Django powered AJAX Chat – Part 1

In the neverending adventure of developing our game website we have stumbled upon the “live chat” rock. I’ve seen some interesting things here using a pure django implementation for chat and there using django and comet for chat. I browsed the first one and I didn’t like it too much, and the comet implementation is clearly out of the picture for now. We have to finish this project in 4 to 6 weeks and we cannot take the risk of messing around with yet another technology (although I will keep this django+comet link bookmarked for future use).

So I decided to make one quick and dirty chat app myself. I had a very clear idea of how I wanted to implement the chat, it is really simple after all:

  1. Client loads HTML + Javascript
  2. Client queries the Server every X seconds for new messages
  3. Server responds messages serialized in JSON format
  4. Client attaches new messages to the message box

This does fail regarding performance, so I did a little bit of reading regarding the matter of creating html based chat applications. It turns out there’s this Comet model to allow persistent or long lived HTTP requests. The idea behind that is simple, for out chat application for instance, instead of quering the server every X seconds just make a request and wait… wait until the server prints SOMETHING into the request. This does save us of making many queries but falls short when implemeting over a normal web server, say Apache. This happens because for every open connection Apache assigns either a thread or a child process to the request. If you know a little about OSs or how the Apache HTTP server works, you would have figured out by now that this implementation can effectivelly occupy all of Apache’s threads discarding any other requests by other clients. FAIL!

To save us from that, diverse solutions exists. Long story short: You can use the Twisted Networking Engine along with Apache, having Apache serve the normal HTTP requests while all other requests (such as our nice chat application) requests are handled by Twisted, which is optimized for this scenarios. Sounds great, doesn’t it? Still, we won’t go this way in the making of this project, not now.

Instead, let’s just make a plain simple server-killer 100% Django chat application! We’ll be needing Django and jQuery.

Heed

I have already posted a follow-up for this post here: https://pythonhaven.wordpress.com/2009/07/13/django-powered-ajax-chat-–-part-2/. It covers the final implementation I used, explaining the application in-depth, and also points you to the google code site where you can get the source code wrapped up in a nice example project. The motivation behind this project is written down in the post you are reading now (part 1), but the implementation is covered in the linked post (part 2).

The models

The philosophy that I have for this application is:

  • Extreme low coupling with the rest of the apps
  • Internationalization capable

Bare with me as I struggle to respect those two aspects.

So first of all we create a chat room model. This will represent a virtual space where messages can be sent and where people can read other people’s messages, the model is as follows:

class Room(models.Model):
    '''Representation of a generic chat room'''
    belongs_to_type = models.CharField(max_length=100, blank=True, null=True)
    belongs_to_id = models.IntegerField(blank=True, null=True)
    created = models.DateTimeField()
    comment = models.TextField(blank=True, null=True)
    objects = RoomManager() # custom manager

    class Meta():
        unique_together = (('belongs_to_type', 'belongs_to_id'))

    def say(self, type, sender, message):
        '''Say something in le chat'''
        m = Message(self, type, sender, message)
        m.save()
        return m

    def messages(self, after=None):
        m = Message.objects.filter(room=self)
        if after:
            m = m.filter(pk__gt=after)
        return m

    def __unicode__(self):
        return 'Chat for %s %d' % (self.belongs_to_type, self.belongs_to_id)

As you have notices this model has two strange, very strange, fields: belongs_to_type and belongs_to_id. I took this off django’s admin login feature. Basically when creating a chat room, you can create a weak reference to ANYTHING you want. For instance let’s say you have project screens and you want each project to have a chatroom of it’s own, you just have to create a room with belongs_to_type = “project” and belongs_to_id = <the id of your project here>. This will then behave like a reverse ForeignKey, instead of having the project point to the chat room, the chat room will point to the project, but, the chat room can also point to… documents, profiles, groups, etc. The possibilities are endless.

Besides that I also defined a custom model manager in the line 06 , I added a couple of utility functions for the creation of rooms and the quick retrieval of rooms:

class RoomManager(models.Manager):
    '''Custom model manager for rooms, this is used for "table-level" operations'''
    def create(self, parent_type, parent_id):
        '''Creates a new chat room and registers it to the calling object'''
        # the first none is for the ID
        r = self.model(None, parent_type, parent_id, datetime.now())
        r.save()
        return r

    def get_room(self, parent_type, parent_id):
        '''Get a room through its parent.'''
        return self.get(belongs_to_type=parent_type, belongs_to_id=parent_id)

I also defined a say function to send a message to the chat room, and a messages function to retrieve the messages of the room.

On to the message, this is the model:

class Message(models.Model):
    '''A message that belongs to a chat room'''
    room = models.ForeignKey(Room)
    type = models.CharField(max_length=1, choices=MESSAGE_TYPE_CHOICES)
    sender = models.CharField(max_length=50, blank=True)
    message = models.CharField(max_length=255)
    timestamp = models.DateTimeField(auto_now=True)

    def __unicode__(self):
        '''Each message type has a special representation, return that representation.
        This will also be translator AKA i18l friendly.'''
        if self.type in ['s','m','n']:
            return u'*** %s' % self.message
        elif self.type == 'j':
            return '*** %s has joined...' % self.sender
        elif self.type == 'l':
            return '*** %s has left...' % self.sender
        elif self.type == 'a':
            return '*** %s %s' % (self.sender, self.message)
        return ''

Simple huh? The message points to the room to which it belongs and then contains a variety of vital information. I was specially interested in the type field, I wanted to have several message types with different representations on the client side, these are the first ones that I thought of:

MESSAGE_TYPE_CHOICES = (
    ('s','system'),
    ('a','action'),
    ('m', 'message'),
    ('j','join'),
    ('l','leave'),
    ('n','notification')
)

So, those are the message types I was interested in having and as you can see the Message object has different representations depending on the message type and they are Internationalization friendly, great!. I am still not sure though on whether the sender field should be a simple CharField or a ForeingKey to django.contrib.auth.models.Users…

Well, that’s all for now, next time we’ll be seeing some jQuery in action to get this thing rolling.

Until next time!

Part 2

Remember, part 2 is here: https://pythonhaven.wordpress.com/2009/07/13/django-powered-ajax-chat-–-part-2/

Advertisements

Fixtures, default data for your models

So, we started working on our project (finally) and although some of the requirements of the project will change in the course of the following days, we are working on the more general aspects of the site. Among these general “apps” is the help app. Basically it has a couple of static pages and then a hierarchy of categories (folders) and topics presented in a similar way to Google’s Gmail Help Page.

This help app is a variation of the usual Polls or Books examples you get in Django’s Writing you first Django App Tutorial, and is a very good exercise to get up to speed with the development in Django. As the app started to take shape we began to wonder if Django provided a mechanism to load default data into the models…

Fortunately it does! You can read all about loading initial data into models it in the docs, but if you want to continue reading about our experiences and choices, read on!

In a nutshell: You can choose between two methods, fixtures or SQL. Fixtures are database agnostic representations of database rows in either XML, JSON, YAML, or Python objects notation. While SQL, you know, consists of SQL insert statements to populate your data back. Both of these can be stored in files and can later be loaded with the magical manage.py script. Obviously we chose fixtures because of their loose coupling with database layer and using XML for maximum compatibility!

So, to generate a fixture containing the current data in the database of the models of an app you only need to run the following manage.py command in the root folder of your Django project:

python manage.py dumpdata <app_name> --format <format>

Where app name is the name of the Django app whose models and format can either be xml, json or yaml. Is you run that command you’ll get the resulting fixture onscreen, that is of no use for us, we need to save it on a file! No worries, you just need to use the output redirection symbols of your console to push that data to a file. So lets say you want to create a fixture for your catalog app…

Do this under GNU/Linux (I assume it is the same under MacOS):

python manage.py dumpdata catalog --format xml > catalog/fixtures/catalog_data.xml

Do this under Windows:

python manage.py dumpdata catalog --format xml > catalog\fixtures\catalog_data.xml

You may have noticed I redirected the output to a fixtures directory inside the app, this is because the Django documentation recommends that for data dumping (using this path is crucial for the last step in this post!)

So, how do you load the data? Simple, use mange.py loaddata!

manage.py loaddata catalog/fixtures/catalog_data.xml

And you are done.

But the whole idea of using fixtures was to provide a simple way to transport data across the team of developers (we are going to be working with SVN, remember?). So reading the doc a little bit more we find that these fixtures can be automatically loaded when performing a syncdb! You only need to save the fixture file with a special name:  initial_data.xml (xml in our case, for you it can be xml, json or yaml) in your app’s fixtures directory.

Now every time you perform a syncdb, the models will be created on the database and the data from the fixture will be automagically loaded, isn’t that great? It is!

Hope this helps!

Until next time.

Join us a djangopeople.net!

Hey, I just stumbled upon a nice “social” site called djangopeople.net. It’s a site listing people who use Django, along with some interesting contact information. As the site says (when i signed up):

Django People lists 4116 Django developers from around the world, in 110 different countries. The aim of the site is to help Django developers find like-minded souls near them, and hopefully kick-start some local meet-ups and user groups.

Come us and join the party!

The tools of choice

Well, we have decided which tools we are going to use for our semester project. The winners are:

Eclipse with the following plugins:

And for our development framework (after a period of uncertainty):

Django was the clear winner.

PostgreSQL will be our DBMS.

Eventually we’ll configure the Apache HTTP Server to work with mod_wsgi for Apache-Python interpretation.

Shortly I’ll post on how to get all those working nice together.

Until then.