Django powered AJAX Chat – Part 2

I’m sorry I left my last Django powered AJAX Chat – Part 1 post so abandoned, we have been dealing with an ever increasing amount of study stuff at the uni and I didn’t have much time to keep on writing about our chat application. I have been working hard to improve some of the functionality of the application, both on the Django side and the jQuery side. There are still many rough edges, but this time I’ve got some more along the lines of plug&play. So, just for those impatient ones who like to see to believe, this is a screenshot of it in action:

Screenshot of the Django chat application using the jQuery Javascript library, you can appreciate the use of special formatting for users who are joining the room, normal formatting for user messages and... yes... smileys!

Pretty neat, huh? There is a lot of ground to cover with this one, so maybe I’ll split this post in two… we will see.

The Idea

Have a pure Django + jQuery chat application working on our site. The users will be presented with a GUI like the one above, listing the messages in the chat room, and will be allowed to submit messages though a text box. A Javascript process on the user’s browser will be quering the server every N seconds for new messages and the server will respond in JSON format, returning a list of the new messages (if any) along with their authors, a timestamp (optional) and an id (very important). If there are new messages, they will be formatted and correctly appended to the chat box. Different types of messages should be supported, among them are: user joins, user leaves, system notifications, special messages, and of course normal user messages. Extras: Try to make it feel as natural as possible, using soft scrolling when new messages arrive, capturing the press of the Enter key when typing a message to automatically send the written text, clear the input box and allow the user to keep on typing… and smilies!

Prerequisites

Before we start, some prerequisites: Django must have the auth and contenttype apps installed, if you are using the Django Admin they are all set up and you can just keep on reading. Rule of thumb: if the admin works, chat will too.

Where Can I Download This and Other Announcements

I though I might as well upload this as a Google Code project (gotta love version control), so you can checkout the project at: http://code.google.com/p/django-jchat/, or download the zip file in the downloads tab. I still strongly recommend that you read my rambles about the code below, for you can gain an overall insight of the code and what I wanted to achieve with each line of code.

Also consider that this is a post I started writing in May… but got hold back until now due to time constraints… so the pasted code could be… a little bit outdated.

The Models

To begin let’s revise the models, as you might recall from the last post we had only two models, a Room model and a Message model. (I didn’t work on a Session model to display the list of users because we already had something like that, so if you want to add one yourself, be my guest, and make sure to drop me a mail on how you modified/improved it :D.)

These models basically behave like your average IRC chat server, you have several chat rooms which users can join and message in. I wanted to make sure my chat app was as portable as possible, so I first worked on a home made solution to allow my Room models to reference different kinds of objects, needless to say that is was sloppy. Unsatisfied with my solution I started looking around for better ones. It didn’t take me long until I stumbled upon Django Admin Log model, which had the capacity of referencing each of my site’s models. Looking into the code I discovered the magic of contenttype’s GenericType and generic.GenericForeignKey classes. They are a work of art! They allow me to establish relationships between my models and any other model with just three extra fields in my models and almost no programming! This is how the Room model looks now:

from django.db import models
from django.contrib.auth.models import User
from django.contrib.contenttypes.models import ContentType, ContentTypeManager
from django.contrib.contenttypes import generic
from datetime import datetime

class RoomManager(models.Manager):
    '''Custom model manager for rooms, this is used for "table-level" operations.
    All methods defined here can be invoked through the Room.objects class.
    @see: http://docs.djangoproject.com/en/1.0/topics/db/managers/#topics-db-managers
    Also see GenericTypes from the contenttypes django app!
    @see: http://docs.djangoproject.com/en/1.0/ref/contrib/contenttypes/''' 

    def create(self, object):
        '''Creates a new chat room and registers it to the calling object'''
        r = self.model(content_object=object)
        r.save()
        return r

    def get_for_object(self, object):
        '''Try to get a room related to the object passed.'''
        return self.get(content_type=ContentType.objects.get_for_model(object), object_id=object.pk)

    def get_or_create(self, object):
        '''Save us from the hassle of validating the return value of get_for_object and create a room if none exists'''
        try:
            return self.get_for_object(object)
        except Room.DoesNotExist:
            return self.create(object)

class Room(models.Model):
    '''Representation of a generic chat room'''
    content_type = models.ForeignKey(ContentType) # to what kind of object is this related
    object_id = models.PositiveIntegerField() # to which instace of the aforementioned object is this related
    content_object = generic.GenericForeignKey('content_type','object_id') # use both up, USE THIS WHEN INSTANCING THE MODEL
    created = models.DateTimeField(default=datetime.now())
    comment = models.TextField(blank=True, null=True)
    objects = RoomManager() # custom manager

    def __add_message(self, type, sender, message=None):
        '''Generic function for adding a message to the chat room'''
        m = Message(room=self, type=type, author=sender, message=message)
        m.save()
        return m

    def say(self, sender, message):
        '''Say something in to the chat room'''
        return self.__add_message('m', sender, message)
    def join(self, user):
        '''A user has joined'''
        return self.__add_message('j', user)

    def leave(self, user):
        '''A user has leaved'''
        return self.__add_message('l', user)

    def messages(self, after_pk=None, after_date=None):
        '''List messages, after the given id or date'''
        m = Message.objects.filter(room=self)
        if after_pk:
            m = m.filter(pk__gt=after_pk)
        if after_date:
            m = m.filter(timestamp__gte=after_date)
        return m.order_by('pk')

    def last_message_id(self):
        '''Return last message sent to room'''
        m = Message.objects.filter(room=self).order_by('-pk')
        if m:
            return m[0].id
        else:
            return 0

    def __unicode__(self):
        return 'Chat for %s %d' % (self.content_type, self.object_id)

from django.db import models

from django.contrib.auth.models import User

from django.contrib.contenttypes.models import ContentType, ContentTypeManager

from django.contrib.contenttypes import generic

from datetime import datetime

class RoomManager(models.Manager):

”’Custom model manager for rooms, this is used for “table-level” operations.

All methods defined here can be invoked through the Room.objects class.

@see: http://docs.djangoproject.com/en/1.0/topics/db/managers/#topics-db-managers

Also see GenericTypes from the contenttypes django app!

@see: http://docs.djangoproject.com/en/1.0/ref/contrib/contenttypes/”’

def create(self, object):

”’Creates a new chat room and registers it to the calling object”’

r = self.model(content_object=object)

r.save()

return r

def get_for_object(self, object):

”’Try to get a room related to the object passed.”’

return self.get(content_type=ContentType.objects.get_for_model(object), object_id=object.pk)

def get_or_create(self, object):

”’Save us from the hassle of validating the return value of get_for_object and create a room if none exists”’

try:

return self.get_for_object(object)

except Room.DoesNotExist:

return self.create(object)

class Room(models.Model):

”’Representation of a generic chat room”’

content_type = models.ForeignKey(ContentType) # to what kind of object is this related

object_id = models.PositiveIntegerField() # to which instace of the aforementioned object is this related

content_object = generic.GenericForeignKey(‘content_type’,’object_id’) # use both up, USE THIS WHEN INSTANCING THE MODEL

created = models.DateTimeField(default=datetime.now())

comment = models.TextField(blank=True, null=True)

objects = RoomManager() # custom manager

def __add_message(self, type, sender, message=None):

”’Generic function for adding a message to the chat room”’

m = Message(room=self, type=type, author=sender, message=message)

m.save()

return m

def say(self, sender, message):

”’Say something in to the chat room”’

return self.__add_message(‘m’, sender, message)

def join(self, user):

”’A user has joined”’

return self.__add_message(‘j’, user)

def leave(self, user):

”’A user has leaved”’

return self.__add_message(‘l’, user)

def messages(self, after_pk=None, after_date=None):

”’List messages, after the given id or date”’

m = Message.objects.filter(room=self)

if after_pk:

m = m.filter(pk__gt=after_pk)

if after_date:

m = m.filter(timestamp__gte=after_date)

return m.order_by(‘pk’)

def last_message_id(self):

”’Return last message sent to room”’

m = Message.objects.filter(room=self).order_by(‘-pk’)

if m:

return m[0].id

else:

return 0

def __unicode__(self):

return ‘Chat for %s %d’ % (self.content_type, self.object_id)

I hope the commentaries are enough for most of you, I don’t want to bore you down with details, and the comments make the code (in my humble opinion) pretty much self explanatory. In any case, just use the comments below to share your inquiries on the code. Don’t despair, I will elaborate on the methods defined at RoomManager later on when we talk about the views.

In any case, the Message model was pretty much unchanged, for the sake of completness I’ll post it here too:

MESSAGE_TYPE_CHOICES = (
    ('s','system'),
    ('a','action'),
    ('m', 'message'),
    ('j','join'),
    ('l','leave'),
    ('n','notification')
)

class Message(models.Model):
    '''A message that belongs to a chat room'''
    room = models.ForeignKey(Room)
    type = models.CharField(max_length=1, choices=MESSAGE_TYPE_CHOICES)
    #sender = models.CharField(max_length=50, blank=True)
    author = models.ForeignKey(User, related_name='author', blank=True, null=True)
    message = models.CharField(max_length=255, blank=True, null=True)
    timestamp = models.DateTimeField(auto_now=True)

    def __unicode__(self):
        '''Each message type has a special representation, return that representation.
        This will also be translator AKA i18l friendly.'''
        if self.type == 's':
            return u'SYSTEM: %s' % self.message
        if self.type == 'n':
            return u'NOTIFICATION: %s' % self.message
        elif self.type == 'j':
            return 'JOIN: %s' % self.author
        elif self.type == 'l':
            return 'LEAVE: %s' % self.author
        elif self.type == 'a':
            return 'ACTION: %s > %s' % (self.author, self.message)
        return self.message

Again it should be somewhat self explanatory ;). If in doubt, just ask.

Views

Phase one complete, we have our pretty nifty models done, now we need to take care of some basic views (and their respective urls) to interact with the chat app. The view itseft does nothing special, it recieves messages sent by users, or returns the list of messages after a certain message, recieves log in and logout events, among other things. I did develop a quick test view (marked in dark orange), and we’ll use it to explain how to create a chat room for your site. But first, the views:

# -*- encoding: UTF-8 -*-
'''
Chat application views, some are tests... some are not
@author: Federico Cáceres <fede.caceres@gmail.com>
'''
from datetime import datetime

from django.http import HttpResponse, Http404
from django.shortcuts import render_to_response
from django.contrib.auth.decorators import login_required
from django.template import RequestContext
from django.contrib.auth.models import User 

from chat.models import Room, Message

@login_required
def send(request):
    '''
    Expects the following POST parameters:
    chat_room_id
    message
    '''
    p = request.POST
    r = Room.objects.get(id=int(p['chat_room_id']))
    r.say(request.user, p['message'])
    return HttpResponse('')

@login_required
def sync(request):
    '''Return last message id

    EXPECTS the following POST parameters:
    id
    '''
    if request.method != 'POST':
        raise Http404
    post = request.POST

    if not post.get('id', None):
        raise Http404

    r = Room.objects.get(id=post['id'])

    lmid = r.last_message_id()    

    return HttpResponse(jsonify({'last_message_id':lmid}))

@login_required
def receive(request):
    '''
    Returned serialized data

    EXPECTS the following POST parameters:
    id
    offset

    This could be useful:
    @see: http://www.djangosnippets.org/snippets/622/
    '''
    if request.method != 'POST':
        raise Http404
    post = request.POST

    if not post.get('id', None) or not post.get('offset', None):
        raise Http404

    try:
        room_id = int(post['id'])
    except:
        raise Http404

    try:
        offset = int(post['offset'])
    except:
        offset = 0

    r = Room.objects.get(id=room_id)

    m = r.messages(offset)

    return HttpResponse(jsonify(m, ['id','author','message','type']))

@login_required
def join(request):
    '''
    Expects the following POST parameters:
    chat_room_id
    message
    '''
    p = request.POST
    r = Room.objects.get(id=int(p['chat_room_id']))
    r.join(request.user)
    return HttpResponse('')

@login_required
def leave(request):
    '''
    Expects the following POST parameters:
    chat_room_id
    message
    '''
    p = request.POST
    r = Room.objects.get(id=int(p['chat_room_id']))
    r.leave(request.user)
    return HttpResponse('')

@login_required
def test(request):
    '''Test the chat application'''

    u = User.objects.get(id=1) # always attach to first user id
    r = Room.objects.get_or_create(u)

    return render_to_response('chat/chat.html', {'js': ['/media/js/mg/chat.js'], 'chat_id':r.pk}, context_instance=RequestContext(request))

def jsonify(object, fields=None, to_dict=False):
    '''Funcion utilitaria para convertir un query set a formato JSON'''
    try:
        import json
    except ImportError:
        import django.utils.simplejson as json

    out = []

    if type(object) not in [dict,list,tuple] :
        for i in object:
            tmp = {}
            if fields:
                for field in fields:
                    tmp[field] = unicode(i.__getattribute__(field))
            else:
                for attr, value in i.__dict__.iteritems():
                    tmp[attr] = value
            out.append(tmp)
    else:
        out = object

    if to_dict:
        return out
    else:
        return json.dumps(out)

(sorry for the cheap template of the blog 😦 )

In any case, the code is quite documented, but I can brief you on the inner workings of each view:

• send: this view recieves the messages sent by users and associates them to the corresponding Room model
• sync: called when the interface loads, basically it requests the id of the last message sent to a chat room, this is used so that the user when joining a chat room will only request messages sent AFTER his arrival
• receive: called by the client’s javacript code, returns a list of messages sent. It uses a variable sent though POST called “offset”, basically it is the id of the last message received by the client (or the id received by the sync view) so that the view only returns messages after it
• join: called when a user joins a chat room
• leave: called when a user leaves a chat room
• test: silly test view, basically allows you to test the app by itself without having to associate it with other apps (more on this on the next parragaph)

In a nutshell this test view gets the Room instance that is related to the user whose id = 1 (in my case this is always the admin user, or the first super user created by the django.contrib.auth app, as it is the first user created when running the syncdb manage script for the first time on a project with the django.contrib.auth app installed.) I am doing this because in my case I know that in the worst scenario, at least I will have a user model to which I can bind this chat room to, so that I can save it and send messages to it…

Based on that test view you can learn how to bind a chat Room to anything you’d like. For example: Imagine that you have a complex site, where you have different user groups. you can enter the page of any particular user group and you would like to display a chat room that is exclusive for that group whose page you are joing. So, if you have a UserGroup model, you could write a view like this to get or create the Room instace associated to your UserGroup model:

from django.shortcuts import render_to_response
from yourapp.chat.models import Room
from models import UserGroup

def usergroup_index(request, group_id):
    group = UserGroup.models.get(id=group_id)
    room = Room.objects.get_or_create(group)
    return render_to_response("your_template.html", {'group':group, 'chat_id':room.id})

Just a quick and dirty example, but basically what that get_or_create manager function does is try to get the Room instance that is associated with the object instance you are passing it as a parameter. It is quite magical thanks again to the GenericType class. This allows you, like I mentioned before, to “attach” or “bind” a chat room to any kind of model.

In any case, it is imperative that your render_to_response call (or whichever method you use) sends the room id to the template (I’ll show you how I use it in the next section) or else nothing will happen!

To wrap this up, this is the urls.py file:

from django.conf.urls.defaults import *

urlpatterns = patterns('',
    url(r'^request/$', 'forum.views.forum'),

    url(r'^$', 'chat.views.test'),
    url(r'^send/$', 'chat.views.send'),
    url(r'^receive/$', 'chat.views.receive'),
    url(r'^sync/$', 'chat.views.sync'),

    url(r'^join/$', 'chat.views.join'),
    url(r'^leave/$', 'chat.views.leave'),
)

With that done, let’s see how a basic template is set up, and the final piece of the puzzle, the Javascript… in front of it all :D.

Template

From now on, you can just rely on the basic copy-paste method, for you should have no need to touch the html (unless you want to modify the looks of it, in which case you should be modifying the css rules). Fortunately this is very easy, just paste the following somewhere in your code:

<div id="chat"></div>

<script type="text/javascript">
$(window).ready(function(){
	init_chat({{ chat_id }}, "chat");
})
</script>

What we see here is a blank <div> block with the chat id. The chat javscript files will place all html elements here for you, so you don’t need to worry. Then we have a short call to a javascript function, using jQuery’s event binding we tell it to “call init_chat” once the window is done loading (html is ready). That “chat_id” variable in the template is the id of the room that your view got from looking up (or creating) a chat Room, and it is necessary so that the javascript managing your chatroom initializes correctly and knows which is the Room it should query.

Of course, you should also include the jquery.js file, you can download it from jquery.com or directly link at a version left available online by the nice people of Google:

<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.3/jquery.min.js"></script>

You shuld place THAT and the next javascript file in your html so that everything works.

Javascript (jQuery)

This is the script done in Javascript using the jQuery Library… documentation to come soon!

var chat_room_id = undefined;
var last_received = 0;

/**
 * Initialize chat:
 * - Set the room id
 * - Generate the html elements (chat box, forms & inputs, etc)
 * - Sync with server
 * @param chat_room_id the id of the chatroom
 * @param html_el_id the id of the html element where the chat html should be placed
 * @return
 */
function init_chat(chat_id, html_el_id) {
	chat_room_id = chat_id;
	layout_and_bind(html_el_id);
	sync_messages();
}

/**
 * Asks the server which was the last message sent to the room, and stores it's id.
 * This is used so that when joining the user does not request the full list of
 * messages, just the ones sent after he logged in.
 * @return
 */
function sync_messages() {
    $.ajax({
        type: 'POST',
        data: {id:window.chat_room_id},
        url:'/chat/sync/',
		dataType: 'json',
		success: function (json) {
        	last_received = json.last_message_id;
		}
    });

	setTimeout("get_messages()", 2000);
}

/**
 * Generate the Chat box's HTML and bind the ajax events
 * @param target_div_id the id of the html element where the chat will be placed
 */
function layout_and_bind(html_el_id) {
		// layout stuff
		var html = '<div id="chat-messages-container">'+
		'<div id="chat-messages"> </div>'+
		'<div id="chat-last"> </div>'+
		'</div>'+
		'<form id="chat-form">'+
		'<input name="message" type="text" class="message" />'+
		'<input type="submit" value="Say"/>'+
		'</form>';

		$("#"+html_el_id).append(html);

		// event stuff
    	$("#chat-form").submit( function () {
            var $inputs = $(this).children('input');
            var values = {};

            $inputs.each(function(i,el) {
            	values[el.name] = $(el).val();
            });
			values['chat_room_id'] = window.chat_room_id;

        	$.ajax({
                data: values,
                dataType: 'json',
                type: 'post',
                url: '/chat/send/'
            });
            $('#chat-form .message').val('');
            return false;
	});
};

/**
 * Gets the list of messages from the server and appends the messages to the chatbox
 */
function get_messages() {
    $.ajax({
        type: 'POST',
        data: {id:window.chat_room_id, offset: window.last_received},
        url:'/chat/receive/',
		dataType: 'json',
		success: function (json) {
			var scroll = false;

			// first check if we are at the bottom of the div, if we are, we shall scroll once the content is added
			var $containter = $("#chat-messages-container");
			if ($containter.scrollTop() == $containter.attr("scrollHeight") - $containter.height())
				scroll = true;

			// add messages
			$.each(json, function(i,m){
				if (m.type == 's')
					$('#chat-messages').append('<div class="system">' + replace_emoticons(m.message) + '</div>');
				else if (m.type == 'm')
					$('#chat-messages').append('<div class="message"><div class="author">'+m.author+'</div>'+replace_emoticons(m.message) + '</div>');
				else if (m.type == 'j')
					$('#chat-messages').append('<div class="join">'+m.author+' has joined</div>');
				else if (m.type == 'l')
					$('#chat-messages').append('<div class="leave">'+m.author+' has left</div>');

				last_received = m.id;
			})

			// scroll to bottom
			if (scroll)
				$("#chat-messages-container").animate({ scrollTop: $("#chat-messages-container").attr("scrollHeight") }, 500);
		}
    });

    // wait for next
    setTimeout("get_messages()", 2000);
}

/**
 * Tells the chat app that we are joining
 */
function chat_join() {
	$.ajax({
		async: false,
        type: 'POST',
        data: {chat_room_id:window.chat_room_id},
        url:'/chat/join/',
    });
}

/**
 * Tells the chat app that we are leaving
 */
function chat_leave() {
	$.ajax({
		async: false,
        type: 'POST',
        data: {chat_room_id:window.chat_room_id},
        url:'/chat/leave/',
    });
}

// attach join and leave events
$(window).load(function(){chat_join()});
$(window).unload(function(){chat_leave()});

// emoticons
var emoticons = {
	'>:D' : 'emoticon_evilgrin.png',
	':D' : 'emoticon_grin.png',
	'=D' : 'emoticon_happy.png',
	':\\)' : 'emoticon_smile.png',
	':O' : 'emoticon_surprised.png',
	':P' : 'emoticon_tongue.png',
	':\\(' : 'emoticon_unhappy.png',
	':3' : 'emoticon_waii.png',
	';\\)' : 'emoticon_wink.png',
	'\\(ball\\)' : 'sport_soccer.png'
}

/**
 * Regular expression maddness!!!
 * Replace the above strings for their img counterpart
 */
function replace_emoticons(text) {
	$.each(emoticons, function(char, img) {
		re = new RegExp(char,'g');
		// replace the following at will
		text = text.replace(re, '<img src="/media/img/silk/'+img+'" />');
	});
	return text;
}

CSS

This is the default css I used:

/* CHAT */

#chat-messages-container {
	overflow: auto; /* used for scrolling */
	width: 100%;
	height: 100px;
	border: 1px solid #777;
	-moz-border-radius: 5px;
}

#chat-messages {
	padding: 5px;
}

#chat-messages .author {
	font-weight: bold;
	margin-right: 0.5em;
	display: inline;
}

#chat-messages .message {
	margin-bottom: 4px;
}

#chat-messages .system {
	font-weight: bold;
	color: #642;
}

#chat-messages .join {
	color: #696;
	background: url(img/silk/door_in.png) top left no-repeat;
	padding-left: 20px;
	margin: 4px 0;
}

#chat-messages .leave {
	color: #966;
	background: url(img/silk/door_out.png) top left no-repeat;
	padding-left: 20px;
	margin: 4px 0;
}

#chat .message {
	width: 80%;
}

The whole mess together

Phew, that was excruciating! Hopefully you got a grasp on how the pieces fit together! But if everything was done correctly, it will work like a charm!

Let me share with you a little extra for reading all the way to the bottom, a little video of me… chatting with my self!

Hope you really enjoyed this, it was an absolute pleasure to work on this app on the marvellous Django framework. As I mentioned on my previous post, there are other ways to do this, which are much more optimized and appropiate, and we also left behind several security and functional issues besides (sending messages to a Room where you have not joined for instance)… but even though of all that, this still works quite well, and is a nice addition to your site. Sure, it may also kill your database if the number of visitors is high enough :P!

Remember you can checkout the whole code from http://code.google.com/p/django-jchat/ it should work right out of the box on any given django project.

And again, if you have questions, just post a comment below and I’ll try to help you out.

Until next time!

• Uncategorized