Fix “import mercurial” problem in windows

15 10 2010

Hi guys, it’s been a while since my last post, this time I want to share a tiny tip that saved me from a huge headache.

At work we’re planning on using Trac to manage our project (mainly as an issue tracker) along with Mercurial. Mercurial is great, all distributed version control systems rock, but when we chose mercurial it just seemed like the right decision due to it’s excellent performance and relative ease to integrate with Apache.

In any case, today I installed trac on our server computer, which unfortunately is a Windows machine. Unfortunate because setting things up on Windows is a bit more troublesome than on Linux or even Mac.

Once Trac was up and running along with our Mercurial server, both under Apache thanks to mod_wsgi, I installed the TracMercurial plugin so that I could setup our Mercurial repository in Trac. But even after adding the repo, I could not even browse the repo.

After reading a couple of posts, I found out that I had to check if the mercurial module could be imported in a normal python prompt. And so

>> import mercurial

… returned a module not found error.

Quick fix

Because I installed Selenic’s official windows binaries, the mercurial module is kept safe in the program files folder, where only the hg.exe binary can access it.

To fix this, I just copied the mercurial folder that was in c:\program files\mercurial\lib\mercurial into Python’s site-packages folder (C:\python26\lib\site-packages when using python 2.6).

And after restarting apache, I could see the “browse source” tab in Trac and all was good!

Note: I know, this is kind of BARBARIC, I could have added mercurial’s path into the PYTHON_PATH, but I just didn’t want to mix all those modules…





Generating Graph Visualizations with pydot and Graphviz

9 12 2009

Hi, for my latest college assignment I had to find a way to visualize data that is interrelated. For instance, my application generated the following data:

A --> B
B --> C
B --> D

And I needed a way to generate pretty graphs without too much headache!

I am already using wxPython for the application’s UI (it saved me a lot of time, and I learned a lot in the process, even implemented my own clone of the Aero Wizard layout used in Windows Vista and 7), so I tried to look for a way to integrate some other super powered library into my app. I quickly remembered a couple of projects that used Graphviz to generate visualizations of Django model definitions: DjangoGraphviz, and django-graphviz. I even hacked one of them to generate visualizations where each app had it’s own color. But these were complicated projects, because they generated some funky XML (correct me if I’m wrong) files that where later fed to graphviz and you had to mess a little with config options.

I wanted something simpler… lucky me, I stumbled into the wonderful project pydot. It was just what I needed, after a couple of experiments I could quickly generate “fantastic” visualizations with almost no extra effort. I’m using this on my Fedora 11 x86_64 box, so I just had to install pydot and easy_install handled the dependencies (it automatically downloaded and installed pyparsing), I am not sure what the requirements for Windows are… specially to get this running with py2exe, I’ll be blogging about that later.

In any case, let me provide you with a quick “getting started” tutorial guide on how this works.

Example 1: “Easy as Pie”

Ok, let’s try an easy one, suppose you have a set of data that you want to represent in a hierarchical way… Say: King, Lords and Vassals… Let’s try to graph that with as little code as we can:

Simple, huh? You should have a graph like this:

Graph generated by example 1

Graph generated by example 1

Now… that looks pretty boring, right? Let’s try something some more… colorful and with a different kind of graph.

Example 2: “Still Easy as Pie”

Ok, on the last example we made an undirected graph, let’s create a directed one now, and let’s also add some colors and labels on the edges between nodes. Say we have four nodes: A, B, C and D, were A points to B, B points to C, C points to D and D points back to A… let’s try that, ok?

That code generates this graph:

Graph generated in example 2

Graph generated in example 2

As you can see, the possibilities are endless, using the attributes you can set for each node, edge and other forms supported by the library, you can easily visualize your data that would otherwise be very hard to see.

One more thing, you can set the attributes when initializing the Node/Edge, or you can use the set_xyz method, for instance, instead of doing this:

pydot.Edge(node_d, node_a, label="and back we go again", labelfontcolor="#009933", fontsize="10.0", color="blue")

You could do this:

edge = pydot.Edge(node_d, node_a)
edge.set_label("and back we go again")
edge.set_labelfontcolor("#009933")
edge.set_fontsize("10.0")
edge.set_color("blue")

That can come in handy.

Well, I hope you found this useful as I have, and that this mini pydot tutorial helps you with this great library.

Don’t forget to visit these links:

Till next time!





wxGrid does not update after a change on PyGridTableBase

1 12 2009

Hi, I have continued my experiments with the fantastic wxPython library.

Today I had to create a simple grid, so I followed some tutorials and read some documentation on how to use wx.grid.Grid along with wx.grid.PyGridTableBase, in a very MVC-ish fashion. The thing is, that I correctly set up both (you can look up tutorials online on how to use them, or the fantastic wxPython in Action book (which was published back in 2006, making it a little bit old, but very a very useful and educational resource nonetheless). But when invoking the AppendRows or DeleteRows on the grid, I saw no change in the Grid’s presentation (although I double checked that the table’s AppendRows and DeleteRows where called accordingly).

But thanks to the wonders of the lazyweb, I found my answer in this very useful post:

http://groups.google.com/group/wxpython-users/msg/f02fb345ef0b35cf

So, all I had to do was to send the corresponding message  on my table’s AppendRows and DeleteRows methods and all worked just fine.

Hope you find that useful too.





How to get XP Styles on wxPython apps made with py2exe

7 10 2009

Let me begin with this: wxPython is fantastic! I could write a moderately simple app in ~17 hours which automates a task that is normally done manually loading data into a web form. The app consists on custom wizard classes, access to web resources (requesting and parsing) and session management with cookies. I’m impressed that I could finish the application, with no previous knowledge on any of the aforementioned subjects, in such a short time span.

Publicity aside, I faced a little problem. I packed the python application into an executable that didn’t require a full python install using the py2exe extension, after reading the guide on how to get py2exe to work with wxPython. My first problem was some silly issue with some MSCV*.dll files:

*** finding dlls needed ***
error: MSVCP90.dll: No such file or directory

After a quick read on several places, I found a simple solution for this problem, ignoring the dll file when building  the executables.

With that solved, I could successfully build and launch my application. The less obvious problem was, that even though the application launched it looked ugly! Here’s a screenshot on what I was getting when running the generated exe vs running the .py file directly:

normal_theme-vs-xp_theme

Hideous indeed. The problem was that for some reason the application was not using XP Themes when drawing the application’s widgets, and after some reading I discovered that there are some serious issues regarding Python2.6 (the version I’m using), wxPython and py2exe when it comes to manifests. Long story short, I had to associate a correct manifest to my exe file, and after lurking around my Python installs, I found a manifest that wxPython creates for Python that served my needs:

<?xml version=’1.0′ encoding=’UTF-8′ standalone=’yes’?>
<assembly xmlns=’urn:schemas-microsoft-com:asm.v1′ manifestVersion=’1.0′>
<trustInfo xmlns=”urn:schemas-microsoft-com:asm.v3″>
<security>
<requestedPrivileges>
<requestedExecutionLevel level=’asInvoker’ uiAccess=’false’ />
</requestedPrivileges>
</security>
</trustInfo>
<dependency>
<dependentAssembly>
<assemblyIdentity
type=’win32′
name=’Microsoft.VC90.CRT’
version=’9.0.21022.8′
processorArchitecture=’*’
publicKeyToken=’1fc8b3b9a1e18e3b’ />
</dependentAssembly>
</dependency>
<dependency>
<dependentAssembly>
<assemblyIdentity
type=”win32″
name=”Microsoft.Windows.Common-Controls”
version=”6.0.0.0″
processorArchitecture=”*”
publicKeyToken=”6595b64144ccf1df”
language=”*” />
</dependentAssembly>
</dependency>
</assembly>
<?xml version='1.0' encoding='UTF-8' standalone='yes'?>
<assembly xmlns='urn:schemas-microsoft-com:asm.v1' manifestVersion='1.0'>
  <trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
    <security>
      <requestedPrivileges>
        <requestedExecutionLevel level='asInvoker' uiAccess='false' />
      </requestedPrivileges>
    </security>
  </trustInfo>
  <dependency>
    <dependentAssembly>
      <assemblyIdentity
     type='win32'
     name='Microsoft.VC90.CRT'
     version='9.0.21022.8'
     processorArchitecture='*'
     publicKeyToken='1fc8b3b9a1e18e3b' />
    </dependentAssembly>
  </dependency>
  <dependency>
    <dependentAssembly>
      <assemblyIdentity
         type="win32"
         name="Microsoft.Windows.Common-Controls"
         version="6.0.0.0"
         processorArchitecture="*"
         publicKeyToken="6595b64144ccf1df"
         language="*" />
    </dependentAssembly>
  </dependency>
</assembly>

Both the “Microsoft.VC90.CRT” and “Microsoft.Windows.Common-Controls” assemblies are important. The first one is mandatory for Python >= 2.6 is built with Visual Studio 2008, and the second one is the one that links our app to the dlls that provide the nice XP Theme widgets.

Just loaded that text into a string (called manifest in my case) in my setup.py file and passed it as an  argument to the setup function like this:

setup(windows=[{
               'script':"app.py",
               'other_resources' : [(24, 1, manifest)]
               }],
        name = "My App",
        version="0.1"
    )

Easy as pie😀.

Hope you find this useful!





Django powered AJAX Chat – Part 2

13 07 2009

I’m sorry I left my last Django powered AJAX Chat – Part 1 post so abandoned, we have been dealing with an ever increasing amount of study stuff at the uni and I didn’t have much time to keep on writing about our chat application. I have been working hard to improve some of the functionality of the application, both on the Django side and the jQuery side. There are still many rough edges, but this time I’ve got some more along the lines of plug&play. So, just for those impatient ones who like to see to believe, this is a screenshot of it in action:

Screenshot of the Django chat application using the jQuery Javascript library

Screenshot of the Django chat application using the jQuery Javascript library, you can appreciate the use of special formatting for users who are joining the room, normal formatting for user messages and... yes... smileys!

Pretty neat, huh? There is a lot of ground to cover with this one, so maybe I’ll split this post in two… we will see.

The Idea

Have a pure Django + jQuery chat application working on our site. The users will be presented with a GUI like the one above, listing the messages in the chat room, and will be allowed to submit messages though a text box. A Javascript process on the user’s browser will be quering the server every N seconds for new messages and the server will respond in JSON format, returning a list of the new messages (if any) along with their authors, a timestamp (optional) and an id (very important). If there are new messages, they will be formatted and correctly appended to the chat box. Different types of messages should be supported, among them are: user joins, user leaves, system notifications, special messages, and of course normal user messages. Extras: Try to make it feel as natural as possible, using soft scrolling when new messages arrive, capturing the press of the Enter key when typing a message to automatically send the written text, clear the input box and allow the user to keep on typing… and smilies!

Prerequisites

Before we start, some prerequisites: Django must have the auth and contenttype apps installed, if you are using the Django Admin they are all set up and you can just keep on reading. Rule of thumb: if the admin works, chat will too.

Where Can I Download This and Other Announcements

I though I might as well upload this as a Google Code project (gotta love version control), so you can checkout the project at: http://code.google.com/p/django-jchat/, or download the zip file in the downloads tab. I still strongly recommend that you read my rambles about the code below, for you can gain an overall insight of the code and what I wanted to achieve with each line of code.

Also consider that this is a post I started writing in May… but got hold back until now due to time constraints… so the pasted code could be… a little bit outdated.

The Models

To begin let’s revise the models, as you might recall from the last post we had only two models, a Room model and a Message model. (I didn’t work on a Session model to display the list of users because we already had something like that, so if you want to add one yourself, be my guest, and make sure to drop me a mail on how you modified/improved it😀.)

These models basically behave like your average IRC chat server, you have several chat rooms which users can join and message in. I wanted to make sure my chat app was as portable as possible, so I first worked on a home made solution to allow my Room models to reference different kinds of objects, needless to say that is was sloppy. Unsatisfied with my solution I started looking around for better ones. It didn’t take me long until I stumbled upon Django Admin Log model, which had the capacity of referencing each of my site’s models. Looking into the code I discovered the magic of contenttype’s GenericType and generic.GenericForeignKey classes. They are a work of art! They allow me to establish relationships between my models and any other model with just three extra fields in my models and almost no programming! This is how the Room model looks now:

from django.db import models
from django.contrib.auth.models import User
from django.contrib.contenttypes.models import ContentType, ContentTypeManager
from django.contrib.contenttypes import generic
from datetime import datetime

class RoomManager(models.Manager):
    '''Custom model manager for rooms, this is used for "table-level" operations.
    All methods defined here can be invoked through the Room.objects class.
    @see: http://docs.djangoproject.com/en/1.0/topics/db/managers/#topics-db-managers
    Also see GenericTypes from the contenttypes django app!
    @see: http://docs.djangoproject.com/en/1.0/ref/contrib/contenttypes/''' 

    def create(self, object):
        '''Creates a new chat room and registers it to the calling object'''
        r = self.model(content_object=object)
        r.save()
        return r

    def get_for_object(self, object):
        '''Try to get a room related to the object passed.'''
        return self.get(content_type=ContentType.objects.get_for_model(object), object_id=object.pk)

    def get_or_create(self, object):
        '''Save us from the hassle of validating the return value of get_for_object and create a room if none exists'''
        try:
            return self.get_for_object(object)
        except Room.DoesNotExist:
            return self.create(object)

class Room(models.Model):
    '''Representation of a generic chat room'''
    content_type = models.ForeignKey(ContentType) # to what kind of object is this related
    object_id = models.PositiveIntegerField() # to which instace of the aforementioned object is this related
    content_object = generic.GenericForeignKey('content_type','object_id') # use both up, USE THIS WHEN INSTANCING THE MODEL
    created = models.DateTimeField(default=datetime.now())
    comment = models.TextField(blank=True, null=True)
    objects = RoomManager() # custom manager

    def __add_message(self, type, sender, message=None):
        '''Generic function for adding a message to the chat room'''
        m = Message(room=self, type=type, author=sender, message=message)
        m.save()
        return m

    def say(self, sender, message):
        '''Say something in to the chat room'''
        return self.__add_message('m', sender, message)
    def join(self, user):
        '''A user has joined'''
        return self.__add_message('j', user)

    def leave(self, user):
        '''A user has leaved'''
        return self.__add_message('l', user)

    def messages(self, after_pk=None, after_date=None):
        '''List messages, after the given id or date'''
        m = Message.objects.filter(room=self)
        if after_pk:
            m = m.filter(pk__gt=after_pk)
        if after_date:
            m = m.filter(timestamp__gte=after_date)
        return m.order_by('pk')

    def last_message_id(self):
        '''Return last message sent to room'''
        m = Message.objects.filter(room=self).order_by('-pk')
        if m:
            return m[0].id
        else:
            return 0

    def __unicode__(self):
        return 'Chat for %s %d' % (self.content_type, self.object_id)
from django.db import models
from django.contrib.auth.models import User
from django.contrib.contenttypes.models import ContentType, ContentTypeManager
from django.contrib.contenttypes import generic
from datetime import datetime
class RoomManager(models.Manager):
”’Custom model manager for rooms, this is used for “table-level” operations.
All methods defined here can be invoked through the Room.objects class.
Also see GenericTypes from the contenttypes django app!
def create(self, object):
”’Creates a new chat room and registers it to the calling object”’
r = self.model(content_object=object)
r.save()
return r
def get_for_object(self, object):
”’Try to get a room related to the object passed.”’
return self.get(content_type=ContentType.objects.get_for_model(object), object_id=object.pk)
def get_or_create(self, object):
”’Save us from the hassle of validating the return value of get_for_object and create a room if none exists”’
try:
return self.get_for_object(object)
except Room.DoesNotExist:
return self.create(object)
class Room(models.Model):
”’Representation of a generic chat room”’
content_type = models.ForeignKey(ContentType) # to what kind of object is this related
object_id = models.PositiveIntegerField() # to which instace of the aforementioned object is this related
content_object = generic.GenericForeignKey(‘content_type’,’object_id’) # use both up, USE THIS WHEN INSTANCING THE MODEL
created = models.DateTimeField(default=datetime.now())
comment = models.TextField(blank=True, null=True)
objects = RoomManager() # custom manager
def __add_message(self, type, sender, message=None):
”’Generic function for adding a message to the chat room”’
m = Message(room=self, type=type, author=sender, message=message)
m.save()
return m
def say(self, sender, message):
”’Say something in to the chat room”’
return self.__add_message(‘m’, sender, message)
def join(self, user):
”’A user has joined”’
return self.__add_message(‘j’, user)
def leave(self, user):
”’A user has leaved”’
return self.__add_message(‘l’, user)
def messages(self, after_pk=None, after_date=None):
”’List messages, after the given id or date”’
m = Message.objects.filter(room=self)
if after_pk:
m = m.filter(pk__gt=after_pk)
if after_date:
m = m.filter(timestamp__gte=after_date)
return m.order_by(‘pk’)
def last_message_id(self):
”’Return last message sent to room”’
m = Message.objects.filter(room=self).order_by(‘-pk’)
if m:
return m[0].id
else:
return 0
def __unicode__(self):

return ‘Chat for %s %d’ % (self.content_type, self.object_id)

I hope the commentaries are enough for most of you, I don’t want to bore you down with details, and the comments make the code (in my humble opinion) pretty much self explanatory. In any case, just use the comments below to share your inquiries on the code. Don’t despair, I will elaborate on the methods defined at RoomManager later on when we talk about the views.

In any case, the Message model was pretty much unchanged, for the sake of completness I’ll post it here too:

MESSAGE_TYPE_CHOICES = (
    ('s','system'),
    ('a','action'),
    ('m', 'message'),
    ('j','join'),
    ('l','leave'),
    ('n','notification')
)

class Message(models.Model):
    '''A message that belongs to a chat room'''
    room = models.ForeignKey(Room)
    type = models.CharField(max_length=1, choices=MESSAGE_TYPE_CHOICES)
    #sender = models.CharField(max_length=50, blank=True)
    author = models.ForeignKey(User, related_name='author', blank=True, null=True)
    message = models.CharField(max_length=255, blank=True, null=True)
    timestamp = models.DateTimeField(auto_now=True)

    def __unicode__(self):
        '''Each message type has a special representation, return that representation.
        This will also be translator AKA i18l friendly.'''
        if self.type == 's':
            return u'SYSTEM: %s' % self.message
        if self.type == 'n':
            return u'NOTIFICATION: %s' % self.message
        elif self.type == 'j':
            return 'JOIN: %s' % self.author
        elif self.type == 'l':
            return 'LEAVE: %s' % self.author
        elif self.type == 'a':
            return 'ACTION: %s > %s' % (self.author, self.message)
        return self.message

Again it should be somewhat self explanatory😉. If in doubt, just ask.

Views

Phase one complete, we have our pretty nifty models done, now we need to take care of some basic views (and their respective urls) to interact with the chat app. The view itseft does nothing special, it recieves messages sent by users, or returns the list of messages after a certain message, recieves log in and logout events, among other things. I did develop a quick test view (marked in dark orange), and we’ll use it to explain how to create a chat room for your site. But first, the views:

# -*- encoding: UTF-8 -*-
'''
Chat application views, some are tests... some are not
@author: Federico Cáceres <fede.caceres@gmail.com>
'''
from datetime import datetime

from django.http import HttpResponse, Http404
from django.shortcuts import render_to_response
from django.contrib.auth.decorators import login_required
from django.template import RequestContext
from django.contrib.auth.models import User 

from chat.models import Room, Message

@login_required
def send(request):
    '''
    Expects the following POST parameters:
    chat_room_id
    message
    '''
    p = request.POST
    r = Room.objects.get(id=int(p['chat_room_id']))
    r.say(request.user, p['message'])
    return HttpResponse('')

@login_required
def sync(request):
    '''Return last message id

    EXPECTS the following POST parameters:
    id
    '''
    if request.method != 'POST':
        raise Http404
    post = request.POST

    if not post.get('id', None):
        raise Http404

    r = Room.objects.get(id=post['id'])

    lmid = r.last_message_id()    

    return HttpResponse(jsonify({'last_message_id':lmid}))

@login_required
def receive(request):
    '''
    Returned serialized data

    EXPECTS the following POST parameters:
    id
    offset

    This could be useful:
    @see: http://www.djangosnippets.org/snippets/622/
    '''
    if request.method != 'POST':
        raise Http404
    post = request.POST

    if not post.get('id', None) or not post.get('offset', None):
        raise Http404

    try:
        room_id = int(post['id'])
    except:
        raise Http404

    try:
        offset = int(post['offset'])
    except:
        offset = 0

    r = Room.objects.get(id=room_id)

    m = r.messages(offset)

    return HttpResponse(jsonify(m, ['id','author','message','type']))

@login_required
def join(request):
    '''
    Expects the following POST parameters:
    chat_room_id
    message
    '''
    p = request.POST
    r = Room.objects.get(id=int(p['chat_room_id']))
    r.join(request.user)
    return HttpResponse('')

@login_required
def leave(request):
    '''
    Expects the following POST parameters:
    chat_room_id
    message
    '''
    p = request.POST
    r = Room.objects.get(id=int(p['chat_room_id']))
    r.leave(request.user)
    return HttpResponse('')

@login_required
def test(request):
    '''Test the chat application'''

    u = User.objects.get(id=1) # always attach to first user id
    r = Room.objects.get_or_create(u)

    return render_to_response('chat/chat.html', {'js': ['/media/js/mg/chat.js'], 'chat_id':r.pk}, context_instance=RequestContext(request))

def jsonify(object, fields=None, to_dict=False):
    '''Funcion utilitaria para convertir un query set a formato JSON'''
    try:
        import json
    except ImportError:
        import django.utils.simplejson as json

    out = []

    if type(object) not in [dict,list,tuple] :
        for i in object:
            tmp = {}
            if fields:
                for field in fields:
                    tmp[field] = unicode(i.__getattribute__(field))
            else:
                for attr, value in i.__dict__.iteritems():
                    tmp[attr] = value
            out.append(tmp)
    else:
        out = object

    if to_dict:
        return out
    else:
        return json.dumps(out)

(sorry for the cheap template of the blog😦 )

In any case, the code is quite documented, but I can brief you on the inner workings of each view:

  • send: this view recieves the messages sent by users and associates them to the corresponding Room model
  • sync: called when the interface loads, basically it requests the id of the last message sent to a chat room, this is used so that the user when joining a chat room will only request messages sent AFTER his arrival
  • receive: called by the client’s javacript code, returns a list of messages sent. It uses a variable sent though POST called “offset”, basically it is the id of the last message received by the client (or the id received by the sync view) so that the view only returns messages after it
  • join: called when a user joins a chat room
  • leave: called when a user leaves a chat room
  • test: silly test view, basically allows you to test the app by itself without having to associate it with other apps (more on this on the next parragaph)

In a nutshell this test view gets the Room instance that is related to the user whose id = 1 (in my case this is always the admin user, or the first super user created by the django.contrib.auth app, as it is the first user created when running the syncdb manage script for the first time on a project with the django.contrib.auth app installed.) I am doing this because in my case I know that in the worst scenario, at least I will have a user model to which I can bind this chat room to, so that I can save it and send messages to it…

Based on that test view you can learn how to bind a chat Room to anything you’d like. For example: Imagine that you have a complex site, where you have different user groups. you can enter the page of any particular user group and you would like to display a chat room that is exclusive for that group whose page you are joing. So, if you have a UserGroup model, you could write a view like this to get or create the Room instace associated to your UserGroup model:

from django.shortcuts import render_to_response
from yourapp.chat.models import Room
from models import UserGroup

def usergroup_index(request, group_id):
    group = UserGroup.models.get(id=group_id)
    room = Room.objects.get_or_create(group)
    return render_to_response("your_template.html", {'group':group, 'chat_id':room.id})

Just a quick and dirty example, but basically what that get_or_create manager function does is try to get the Room instance that is associated with the object instance you are passing it as a parameter. It is quite magical thanks again to the GenericType class. This allows you, like I mentioned before, to “attach” or “bind” a chat room to any kind of model.

In any case, it is imperative that your render_to_response call (or whichever method you use) sends the room id to the template (I’ll show you how I use it in the next section) or else nothing will happen!

To wrap this up, this is the urls.py file:

from django.conf.urls.defaults import *

urlpatterns = patterns('',
    url(r'^request/$', 'forum.views.forum'),

    url(r'^$', 'chat.views.test'),
    url(r'^send/$', 'chat.views.send'),
    url(r'^receive/$', 'chat.views.receive'),
    url(r'^sync/$', 'chat.views.sync'),

    url(r'^join/$', 'chat.views.join'),
    url(r'^leave/$', 'chat.views.leave'),
)

With that done, let’s see how a basic template is set up, and the final piece of the puzzle, the Javascript… in front of it all😀.

Template

From now on, you can just rely on the basic copy-paste method, for you should have no need to touch the html (unless you want to modify the looks of it, in which case you should be modifying the css rules). Fortunately this is very easy, just paste the following somewhere in your code:

<div id="chat"></div>

<script type="text/javascript">
$(window).ready(function(){
	init_chat({{ chat_id }}, "chat");
})
</script>

What we see here is a blank <div> block with the chat id. The chat javscript files will place all html elements here for you, so you don’t need to worry. Then we have a short call to a javascript function, using jQuery’s event binding we tell it to “call init_chat” once the window is done loading (html is ready). That “chat_id” variable in the template is the id of the room that your view got from looking up (or creating) a chat Room, and it is necessary so that the javascript managing your chatroom initializes correctly and knows which is the Room it should query.

Of course, you should also include the jquery.js file, you can download it from jquery.com or directly link at a version left available online by the nice people of Google:

<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.3/jquery.min.js"></script>

You shuld place THAT and the next javascript file in your html so that everything works.

Javascript (jQuery)

This is the script done in Javascript using the jQuery Library… documentation to come soon!

var chat_room_id = undefined;
var last_received = 0;

/**
 * Initialize chat:
 * - Set the room id
 * - Generate the html elements (chat box, forms & inputs, etc)
 * - Sync with server
 * @param chat_room_id the id of the chatroom
 * @param html_el_id the id of the html element where the chat html should be placed
 * @return
 */
function init_chat(chat_id, html_el_id) {
	chat_room_id = chat_id;
	layout_and_bind(html_el_id);
	sync_messages();
}

/**
 * Asks the server which was the last message sent to the room, and stores it's id.
 * This is used so that when joining the user does not request the full list of
 * messages, just the ones sent after he logged in.
 * @return
 */
function sync_messages() {
    $.ajax({
        type: 'POST',
        data: {id:window.chat_room_id},
        url:'/chat/sync/',
		dataType: 'json',
		success: function (json) {
        	last_received = json.last_message_id;
		}
    });

	setTimeout("get_messages()", 2000);
}

/**
 * Generate the Chat box's HTML and bind the ajax events
 * @param target_div_id the id of the html element where the chat will be placed
 */
function layout_and_bind(html_el_id) {
		// layout stuff
		var html = '<div id="chat-messages-container">'+
		'<div id="chat-messages"> </div>'+
		'<div id="chat-last"> </div>'+
		'</div>'+
		'<form id="chat-form">'+
		'<input name="message" type="text" class="message" />'+
		'<input type="submit" value="Say"/>'+
		'</form>';

		$("#"+html_el_id).append(html);

		// event stuff
    	$("#chat-form").submit( function () {
            var $inputs = $(this).children('input');
            var values = {};

            $inputs.each(function(i,el) {
            	values[el.name] = $(el).val();
            });
			values['chat_room_id'] = window.chat_room_id;

        	$.ajax({
                data: values,
                dataType: 'json',
                type: 'post',
                url: '/chat/send/'
            });
            $('#chat-form .message').val('');
            return false;
	});
};

/**
 * Gets the list of messages from the server and appends the messages to the chatbox
 */
function get_messages() {
    $.ajax({
        type: 'POST',
        data: {id:window.chat_room_id, offset: window.last_received},
        url:'/chat/receive/',
		dataType: 'json',
		success: function (json) {
			var scroll = false;

			// first check if we are at the bottom of the div, if we are, we shall scroll once the content is added
			var $containter = $("#chat-messages-container");
			if ($containter.scrollTop() == $containter.attr("scrollHeight") - $containter.height())
				scroll = true;

			// add messages
			$.each(json, function(i,m){
				if (m.type == 's')
					$('#chat-messages').append('<div class="system">' + replace_emoticons(m.message) + '</div>');
				else if (m.type == 'm')
					$('#chat-messages').append('<div class="message"><div class="author">'+m.author+'</div>'+replace_emoticons(m.message) + '</div>');
				else if (m.type == 'j')
					$('#chat-messages').append('<div class="join">'+m.author+' has joined</div>');
				else if (m.type == 'l')
					$('#chat-messages').append('<div class="leave">'+m.author+' has left</div>');

				last_received = m.id;
			})

			// scroll to bottom
			if (scroll)
				$("#chat-messages-container").animate({ scrollTop: $("#chat-messages-container").attr("scrollHeight") }, 500);
		}
    });

    // wait for next
    setTimeout("get_messages()", 2000);
}

/**
 * Tells the chat app that we are joining
 */
function chat_join() {
	$.ajax({
		async: false,
        type: 'POST',
        data: {chat_room_id:window.chat_room_id},
        url:'/chat/join/',
    });
}

/**
 * Tells the chat app that we are leaving
 */
function chat_leave() {
	$.ajax({
		async: false,
        type: 'POST',
        data: {chat_room_id:window.chat_room_id},
        url:'/chat/leave/',
    });
}

// attach join and leave events
$(window).load(function(){chat_join()});
$(window).unload(function(){chat_leave()});

// emoticons
var emoticons = {
	'>:D' : 'emoticon_evilgrin.png',
	':D' : 'emoticon_grin.png',
	'=D' : 'emoticon_happy.png',
	':\\)' : 'emoticon_smile.png',
	':O' : 'emoticon_surprised.png',
	':P' : 'emoticon_tongue.png',
	':\\(' : 'emoticon_unhappy.png',
	':3' : 'emoticon_waii.png',
	';\\)' : 'emoticon_wink.png',
	'\\(ball\\)' : 'sport_soccer.png'
}

/**
 * Regular expression maddness!!!
 * Replace the above strings for their img counterpart
 */
function replace_emoticons(text) {
	$.each(emoticons, function(char, img) {
		re = new RegExp(char,'g');
		// replace the following at will
		text = text.replace(re, '<img src="/media/img/silk/'+img+'" />');
	});
	return text;
}

CSS

This is the default css I used:

/* CHAT */

#chat-messages-container {
	overflow: auto; /* used for scrolling */
	width: 100%;
	height: 100px;
	border: 1px solid #777;
	-moz-border-radius: 5px;
}

#chat-messages {
	padding: 5px;
}

#chat-messages .author {
	font-weight: bold;
	margin-right: 0.5em;
	display: inline;
}

#chat-messages .message {
	margin-bottom: 4px;
}

#chat-messages .system {
	font-weight: bold;
	color: #642;
}

#chat-messages .join {
	color: #696;
	background: url(img/silk/door_in.png) top left no-repeat;
	padding-left: 20px;
	margin: 4px 0;
}

#chat-messages .leave {
	color: #966;
	background: url(img/silk/door_out.png) top left no-repeat;
	padding-left: 20px;
	margin: 4px 0;
}

#chat .message {
	width: 80%;
}
The whole mess together

Phew, that was excruciating! Hopefully you got a grasp on how the pieces fit together! But if everything was done correctly, it will work like a charm!

Let me share with you a little extra for reading all the way to the bottom, a little video of me… chatting with my self!

Hope you really enjoyed this, it was an absolute pleasure to work on this app on the marvellous Django framework. As I mentioned on my previous post, there are other ways to do this, which are much more optimized and appropiate, and we also left behind several security and functional issues besides (sending messages to a Room where you have not joined for instance)… but even though of all that, this still works quite well, and is a nice addition to your site. Sure, it may also kill your database if the number of visitors is high enough :P!

Remember you can checkout the whole code from http://code.google.com/p/django-jchat/ it should work right out of the box on any given django project.

And again, if you have questions, just post a comment below and I’ll try to help you out.

Until next time!





Find relation between two models

24 05 2009

Today one of my teammates and I were working on a complex SQL query which was beyond Django’s ORM normal API, so we had to use the extra method to build the query. After running with a couple of issues with PostgreSQL’s GROUP BY we decided to opt quickly for subquieries, and in no time we had our query done. The problem was though, that we had to do this operation on several models that where similar to each other, the only thing that changed was the WHERE clause depending on the models involved.

So I set off so find a way to automatically discover which was the relation between two models. In short time I discovered this: MyModel._meta.get_all_related_objects(). This returns a list of RelatedObject instances that tell us about which models point to MyModel  though a foreign key, cool! So, let us suppose we have a Car model and a Rental model. The Rental model has a ForeingKey to Car, to know how they are related I built something like this:

for r in target_model._meta.get_all_related_objects():
if r.model == ranking_model:
relation = r
for r in Car._meta.get_all_related_objects():
    if r.model == Rental:
        relation = r
        break

now in relation I have a RelatedObject (I cannot seem to find a lot of information about this on the net) instace, which has very useful fields like:

  • parent_model: the model that is being refered in this relation (in this case Car)
  • model: the Model (in this case Rental) that points to the parent model (in this case Car)
  • field: the field in model that references to parent_model

So now I know that if I want to build a custom SQL in Django and I really want to make it generic as possible, I can use this RelatedObject class along with other interesting model attributes like: Model._meta.db_table (getting the name that the model has on your database) and such to make my code 100% portable😀 (or at least try to).

Just for the fun of it, this is the resulting code (not that it will make much sense to you… and sorry for the lousy formatting):

return target_model.objects.extra(select={"sum":"SELECT SUM(points) "
                                                "FROM %s "
                                                "WHERE date_part('year', reg_date) = %d "
                                                "AND %s.%s = %s.%s" % (ranking_model._meta.db_table, year, # year
                                                                       ranking_model._meta.db_table, # ranking table name
                                                                       relation.field.column, # column on ranking that points to user/team table
                                                                       target_model._meta.db_table, # user/team table
                                                                       target_model._meta.pk.column)}, # id column on user/team table
                                  order_by=['sum'])[:10]

Use with wisdom!

Now, if there is an easier way to do this, please tell me, I’m eager to learn!





Configuring Apache + mod_wsgi + django… on Virtual Box serving files from Windows 7!

24 05 2009

If you enjoy reading about weird software combinations let me tell you about my system configuration. I am currently using Windows 7 RC as my primary development OS (I quickly replaced my XP SP3) and I am finding it quite good to use (waiting for that new Fedora with the lovely Kde 4.2 to come out). In any case. I wanted to test how to set up Apache on a linux machine, but I didn’t want to go through the hassle of partitioning my HDD (not yet) so I decided to try Sun’s Virtual Box virtualization solution.

I got my Fedora 10 distro up and running in a jiffy, including the “Guest additions” that worked right out of the box (something that never really worked for me with VMWare). So, soon I started installing all the necessary packages for our project:

yum install Django

yum install python-psycopg2

yum install python-markdown

yum install  python-dateutil

yum install mod_wsgi

I did have to download and install the PIL library by hand because I didn’t find the appropiate package for fedora and I already knew I could downlaod the tar.gz file from their site and use the infamous setup.py install command to install it.

The thing is, I want to keep my development files in sync with the files that were going to be served by Apache, that is, find a way to mount some sort of virtual drive connecting the virtualzied Fedora with the Windows 7 host. Thanks to Virtual Box’s shared folders options this was a breeze!

So I set up a shared folder on vbox’s shared folders menu, I pointed that to the trunk dir of our project and named it “mgtrunk”.

I refuse to mount this folder by hand each time, so I modified my fstab file and added a line like this at the end:

mgtrunk /mnt/mg vboxsf defaults 0 0
mgtrunk /mnt/mg vboxsf defaults 0 0

Obviously I created the “mg” folder at the “mnt” directory previosly🙂.

A quick reboot and everything was good to go.

The next step was configuring mod_wsgi (which was automatically generated by the yum installed), so I went on and edited the wsgi.conf file created under /etc/httpd/conf.d/

$ vim /etc/httpd/conf.d/wsgi.conf

I edited it adding lines like the ones in mod_wsgi’s integration with django wiki page relating to the Apache config. And also created a django.wsgi file in my django project under the apache/ dir, just like the aformentioned page mentions.

With a lot of expectations I started the httpd service, and rushed to http://localhost/ on my browser… BOOM… Cannot load the page, followed by a pesky SELinux ballon hovering on the screen. What to do? Disable SELinux through it’s administrative interface, I have no time tonight to mess with yer configs (I did try… chcon’ing some files… but errors just kept on appearing from nowhere! So I just disabled SELinux).

And after another reboot I got better results, the page was loading, but unfortunately no images or css where loaded. Apache was not service correctly images no css files or other files, the only thing that worked was the python scripts. Quickly I changed my config to check if an unmodified apache config worked (I overwrote the / directory with my mod_wsgi after all) and images were displayed correctly, how odd! I even uninstalled the mod_python extension to check if it wasn’t causing any conflicts! No luck…

Man this did take me time!  As it turns out, Apache uses some sort of wicked kernel file transfer functionality to directly transfer files. This is how it reads on the httpd.conf file:

# EnableSendfile: Control whether the sendfile kernel support is

# used to deliver files (assuming that the OS supports it).

# The default is on; turn this off if you serve from NFS-mounted

# filesystems.  Please see

# http://httpd.apache.org/docs/2.2/mod/core.html#enablesendfile

Hmm… NFS… Network File System…. I even wonder if the vboxsf supports this sendfile functionality, maybe it doesn’t. So all it took me for the files to display correctly was DISABLING the darn sendfile directive. I added a line on my wsgi.conf file, just before the Directory tag for the media folder, that reads like this:

EnableSendfile off

A final service httpd restart…. and… yay! The site’s working correctly, at last!

Quite an adventure… Now on I can go to try to develop a super scalable WSGI application to interface with my Django site like Eric Florenzano’s Writing Blazing Fast, Infinitely Scalable, Pure-WSGI Utilities blog article.








Follow

Get every new post delivered to your Inbox.