Posts Tagged ‘django’

« Older Entries

Developing Reusable Django Apps: Signals

Thursday, March 4th, 2010

I wrote “signals provide a great way to propagate the events generated from your app” earlier. I think reusable apps should avoid hardcoding any kind of event handling and send signals instead. App consumers might prefer an email over an on-screen notification. They may even choose to ignore the event silently. A reusable app should give this choice to the consumer.

Taking advantage of signals doesn’t necessarily mean providing no sane defaults. You can send signals and provide default event handling. Here is a couple of ideas how this can be done:

  • Your app can check if there are any listeners and connect the default handlers if there is none.
  • You can ship an auxiliary app that connects default handlers when added to INSTALLED_APPS.

I personally prefer the second approach since it’s simpler and more explicit. I’m sure there are other ways to implement default handlers for signals.

Dispatch_uid

Don’t forget to assign a unique dispatch_uid for each connect() call. Otherwise your handler can get connected twice. I would also suggest you to use both module path and your handler function’s name in your dispatch_uid:

"%s.%s" % (os.path.splitext(__file__)[0].replace(os.sep, '.')[1:],
           handler_name)

Now I should take my own advice and replace hardcoded User.message_set.create()s with signals in django-simple-friends.

Bookmark and Share

Tags: , , ,
Posted in Programming | 5 Comments »

Web Site Performance Optimizations

Monday, February 22nd, 2010

Recently I have done some optimizations to make telvee a little faster using django_compressor and making sprites for background images. Good news is substantial changes to development environment and the design wasn’t required. I’ll get into details below. But first I’d like to write about the theory a little bit.

I follow (and read with great interest) Steve Souders’s blog High Performance Web Sites. I must admit I was sceptical about it at first; spriting images, individual different loading behaviours of browsers… I thought they were premature optimizations. But I realized I was wrong as I continued to read. Steve Souders is an expert on high performance web sites and what he preaches are realistic techniques, backed by test results most of the time. If you are not following, I suggest you add it to your RSS reader.

Optimization Techniques

We can explore the techniques in two main categories:

  1. Techniques to reduce the data to be transferred.
    • Minifying: Minification is removing comments and unneeded whitespace in CSS and JavaScript files. Compilers like YUI Compressor and Closure modifies JavaScript code to compress even further, without any changes to the functionality.
    • Gzipping: Web browsers accept gzip encoded content for a long time. I have just compressed a 13 KB text file down to 4 KB. A two thirds compression ratio is not bad at all.
  2. Techniques to reduce the number of HTTP requests.
    • Combining: CSS and JavaScript files can be combined together into a single file and therefore a single HTTP request. Gzip may be more efficient on these files. In a similar way background images can be merged into a sprite and then reconstructed again using their coordinates on that sprite.
    • Data URI’s: Images (or other file types) can be embedded into CSS (or JavaScript or HTML) using data: URI’s. Extra HTTP requests for those resources can be avoided this way.

You might think it’s fine to perform all these optimizations, but what happens when I want to make some changes to my combined, minified JavaScript file? Instead of applying these techniques blindly, it’s best to follow a sensible plan for implementing these optimizations:

  • First of all everything that can be automated should be automated. Regarding the example above script files should stay uncombined and uncompressed in the development environment and optimizations should be applied when the application is published. Taking it a step further we can have the application detect changes in those files and update optimized versions automatically. (django_compressor works this way)
  • I was worried that spriting would complicate managing the design. But I have seen, on the contrary; if images each sprite contain are choosen carefully it makes the process easier. Start combining images that belong to the same design element. Avoid complex arrangements, stick with horizontal or vertical stacking as much as possible. Don’t forget to leave transparent spaces between items and sprite border. Try to combine images that are loaded on the same page, avoid loading a sprite for only half of it’s elements. Don’t force yourself to combine all images, if you follow the guidelines I have mentione they won’t.
  • When performing optimizations don’t forget to use easily available tools. You can use YSlow for general analysis, SpriteMe for image combining tips, CompressorRater to compare different compilers’ performance on your scripts. I would like to note that Steve Souders is the developer of first two.

Telvee Results

I didn’t think about performance at all when I started developing telvee. Too many CSS files and too many images were being loaded. Here is what it looked like before optimizations:

 # of requestsload (KB)
Homepage25~85
Cup Detail48~80

Then I have installed and configured django_compressor. I used YUI Compressor for both JavaScript and CSS. I have created sprites and modified CSS files manually1. Then I deployed these changes and measured again:

 # of requestsload (KB)
Homepage12~70 (~160)
Cup Detail14~64

In the load column of Homepage, the number in parens is the actual load. But the design of homepage is changed with this upgrade and a new 90 KB image is being loaded now. So I have accepted 70 KB in my calculations. The result of optimizations are as follows:

 # of requestsload (KB)
Homepage52%17%
Cup Detail70%19%

Django_compressor and Data URI’s

Django_compressor, developed by Christian Metts, helps you to apply optmizations I have mentioned above easily to your Django projects. You can see my fork here where I have merged some other branches and added a little bit of code myself.

Using compressor.filters.datauri.CssDataUriFilter in data-uri branch of this repository, you can embed linked files within your CSS files. It will only embed files less than or equal to 1024 Bytes (1 KB) by default. You can change this limit by setting COMPRESS_DATA_URI_MIN_SIZE in your settings.py.

There are a couple of things to pay attention when you convert your references to data: URIs. Firstly file contents are base64 encoded which means approximately one third increase in size. It’s up to you to balance between increased bandwidth and reduced request counts2. Another thing to watch for is multiple references to the same file will end up embedding the same data many times. The solution to this problem is to reduce all references to one3 but this might break your CSS arrangement strategy.

Please test django_compressor’s data: URI support and tell me what you think. If you haven’t applied optimizations I mentioned above, you should. Thanks to django_compressor they are quite easy to implement on Django projects.

1: I would like to add automatic sprite building/linking support to django_compressor sometime.

2: With Today’s modern connections 1~2 KB increase is a good price for 1 less HTTP request..

3: http://meiert.com/en/blog/20090401/why-css-needs-no-variables/

Bookmark and Share

Tags: , , , ,
Posted in Internet, Programming | 3 Comments »

Developing Reusable Django Apps: App Settings

Tuesday, January 26th, 2010

Eventually your app will need some sort of configuration. Supplying many parameters to customise your views, template tags and filters to allow template authors easily harness the power of your app and implementing registration pattern are all sensible things to do. But at some point you will need configuration for your app at project level. Because app level configuration is stupidnot reusable. Consumers of your app should never have to change its source code. And what better place to put our configuration statements than settings.py? Remember; we want to make things easier for our app’s consumers, not harder. There is no need to add a new file to the project for a few lines of settings1.

We already know we shouldn’t import settings.py directly. Instead we import the settings object (much like a singleton) of django.conf module:

from django.conf import settings

Now you can access different configuration options as attributes on this object. But I suggest you to use getattr() in order to avoid getting AttributeErrors. Also, notice how we didn’t hardcode the name of the attribute in the second method below:

# This is too verbose
try:
    some_setting = settings.SOME_SETTING
except AttributeError:
    some_setting = DEFAULT_VALUE

# Plain and simple
some_setting = getattr(settings, 'SOME_SETTING', DEFAULT_VALUE)

Instead of requiring consumers to define all your app settings it is better to supply sensible defaults. Also I find it useful to prefix names of app settings within settings.py.

# in settings.py
MYAPP_FOO_CHOICES = [('bar', u'Bar'), ('baz', u'Baz')]


# in myapp/models.py
from django.db import models
from django.conf import settings


FOO_CHOICES = getattr(settings, 'MYAPP_FOO_CHOICES', [('quux', u'Quux')])


class FooRecord(models.Model):
    foo = models.CharField(max_length=10, choices=FOO_CHOICES)

This works fine for simple apps with fewer settings. But it can easily get out of hand when your app grows. An app_settings.py module would help keeping track of configuration by keeping all configuration options in one place:

# in myapp/app_settings.py
from django.conf import settings


FOO_CHOICES = getattr(settings, 'MYAPP_FOO_CHOICES', [('quux', u'Quux')])


# in myapp/models.py
from django.db import models
from app_settings import FOO_CHOICES


class FooRecord(models.Model):
    foo = models.CharField(max_length=10, choices=FOO_CHOICES)

To summarize the points above:

  • Import settings from django.conf
  • Use getattr()
  • Always supply a default value.
  • Prefix settings you made up in settings.py
  • Use app_settings.py if you have many

1: If your configuration is long, say more than 100 lines, you should step back and reconsider. Perhaps you should prefer a strategy similar to django.contrib.sitemaps or django.contrib.syndication.

Bookmark and Share

Tags: , , , ,
Posted in Programming | 1 Comment »

Developing Reusable Django Apps

Wednesday, January 13th, 2010

Django app structure is an implementation of seperation of concerns. It is slightly different than what you can find in other MVC frameworks. The stack is split vertically, not horizontally. And then the app is split horizontally within, i.e. models, views, templates etc are in their seperate modules/packages/directories. This vertical splitting allows you to collect all ingredients of one functionality in your project in one place.

Framework Structure

I think apps1 are one of the strong points of Django. A selling point if you like. There is a great ecosystem of apps, you can find an app for almost anything posssible with Python. And Python is kick-ass when it comes to library wealth. But there is another major advantage of apps when they’re done right; a sane code base. Here is a slide from the Django in the Real World presentation by Jacob Kaplan-Moss:

The fivefold path

  • Do one thing, and do it well.
  • Don’t be afraid of multiple apps.
  • Write for flexibility.
  • Build to distribute.
  • Extend carefully.

I will focus on flexibility and interoperability of apps in this post. But before we proceed I would like to emphasize the first bullet point in the slide. Because the scope of your app plays a big role in its flexibility and interoperability. Apps should be small enough to easily understand and integrate (into a project). Many times I have moved away from an otherwise good app because of its many dependencies and/or excessive features. On the other hand apps should be big enough to allow for different configurations and allow extension without modifying their code. Do one thing, and do it well.

Scope of an app

Take django-tagging for example; it’s 1.3 KLOC but it does tagging and nothing else. There are no dependencies other than Django, you can add tags to any model without modifying the model source, a tag can be associated with any type of model and tagging hides the gory details from you… In short; finding the right size is important. This is why tagging is the tagging app for Django.

Building For Reuse

General advice is “even project specific apps should be reusable“. Slapping the same app onto another project is not the only advantage. In fact it may not be possible if you are not in the habit of upgrading your whole project to recent versions of Django. The main advantage as I have said before is sanity. I prefer Django to other web frameworks/environments because it provides a civilized way of development. Let’s accept it; web programming is not a particularly interesting, exciting or intellectually rewarding field. You write the same piece of code over and over. And worst of all the challanges you face are actually a result of either the underlying system was designed by morons or you are trying to use it for something it’s not intended to be used. So it is only natural that web programmers feel they’re rusting. Django eases the pain. If you stick to certain conventions serenity will follow as well.

Naturally the framework does most of the work regarding app flexibility and interoperability. Take URLs for instance include('myapp.urls') and you are good to go. You don’t have to bind views one by one. Is it inflexible? Who said urls.py can only contain a hardcoded list of URLs. You can do anything that is possible with Python. You can generate different urlpatterns based on a setting for instance.

It is relatively easy and straightforward to reuse and extend forms and views (both function based and class based). Models are a little harder to get right though. You should always think of the most difficult situation which is you can’t touch either app’s code. Registration pattern of admin app provides a good solution here. You can register a third party model to another third party app in just a few lines.

You don’t need to write lots of code to get the flexibility and interoperability. Well designed apps make good use of settings.py for example. Why should the project developer wrap a view when a single line assignment would do the job? Supplying good templatetags and template snipplets (includes) is another way to make things easy for app consumers.

Signals provide a great way to propagate the events generated from your app. Even though they are one way2, signals are extremely powerful. Any number of observers can connect to a signal and you can send a signal anywhere in your code. Literally. It is even possible your app suppying a signal and then another app sending it3.

There are many more ways to tame your app to be reusable. It all starts with your determination and discipline. Just like documentation, testing and maintaining a software generally. I will write more about reusable apps.

1: The word application is used both for a web application and a Django application. To avoid confusion I always use app to indicate the latter.

2: Signals don’t have return values. But you can use a callback AFAIK.

3: I can’t think of an example this would be useful, but still…

Bookmark and Share

Tags: , , ,
Posted in Programming | Comments Off

Dynamic Translation Apps for Django

Wednesday, January 6th, 2010

When I needed multi-language flatpages and flatblocks for telvee I searched for available Django apps that do dynamic translation. By dynamic translation, I mean translations are entered and stored in the database. As I said I needed to be able to translate both full pages and chunks of text that I can include into another page. I ended up rolling my own, which is without a doubt lesser to some apps below. I will try to imrove it with the good ideas from existing projects and then open source. Meanwhile I would like to share my review of 6 dynamic translation apps with you. I hope someone out there finds it useful.

Django-multilingual

One of the few active projects I have reviewed is django-multilingual. To enable a model for translations you need to create a Translation inner-class and move translatable fields inside. Then a seperate model for those fields is created behind the scenes. Django-multilingual has admin integration and a multi-language flatpages app. Another nice thing about this app is that it’s using signals internally. The API is based on getter and setter methods. For instance you call get_<fieldname>(language_id=None) on your model to get the field value for the active translation. This app has no releases and less than satisfactory API documentation and has tests only on the example project.

Django-pluggable-model-i18n

Django-pluggable-model-i18n uses registration pattern of admin app. It creates an extra model for translated fields and stores translations of non-default languages in it. This app has no releases, no tests, no API documentation and it is clearly stated to be experimental. I would also like to note the last commit date is May 25, 2009.

Django-modeltranslation

Another app that implements registration pattern is django-modeltranslation. The advantage of registration pattern is you don’t have to modify the code of the 3rd party apps you want to translate dynamically. But the database schema still needs to be modified if you are using django-modeltranslation. Also there is one translations.py for the project. I think one translation definition file per app would be better (just like admin does). Django-modeltranslation has intuitive underscore-language_code API but it is somewhat inconsistent; when you read the unsuffixed field you read the currently active language, when you write you write the default language specified in settings.py. Also the modified model stores some redundant data. For instance if your DEFAULT_LANGUAGE is "ES" both <fieldname> and <fieldname>_es columns will store the same value. Django-modeltranslation has admin integration and a management command to update database schema. Most importantly this app is the only one which has both tests and a release. (Unfortunately tests require specific settings to run)

Transdb

Transdb takes a completely different approach to dynamic translation problem. It provides two new field types; TransCharField and TransTextField. Then it serializes all your translations within a single column for each field of those. This means no JOINs and no extra queries. Unfortunately transdb doesn’t implement underscore-language_code API, you need to use get_in_language() and set_in_language() methods. Transdb has default widgets that render one form field for each language. Last commit date is Nov 07, 2008 and there is a release.

Django-multilingual-model

This one is not actually an app but just one module with 33 42 lines of code. You need to define the model that holds translations manually. This introduces some code redundancy, since you also define which fields get translated in the original model. Django-multilingual-model doesn’t implement any translation API, so it’s rather verbose to do anything with it. There are no tests and no releases. I simply don’t recommend django-multilingual-model for anything serious.

Django-transmeta

Django-transmeta stores translations in extra columns it creates in the original field’s table similar to django-modeltranslation. But you enable translation assigning a metaclass for your model and then add a Meta attribute; this means you can’t make models in existing apps translatable without modifying their code. Django-transmeta implements underscore-language_code API, has admin integration and a management command to sync database when you add new languages or translatable fields. There are documentation and code examples but no tests or releases. Last commit date it Nov 24, 2009.

Comparison of Dynamic Translation Apps

Comparison of Dynamic Translation Apps

Software development is making choices. Would you rather have a clean and stable schema with an extra translations model or avoid extra JOINs and denormalize translations onto your original model’s table? Both have advantages and disadvantages. But some choices are not based on trade-offs. Documentation, examples, tests and releases for instance. Also in my opinion underscore-language_code API is way better than any of the alternatives.

Django platform is a very powerful and intuitive one. Many people have moved in last year. This popularity affected app ecosystem as well. But unfortunately a significant number of those apps are half baked fire-and-forget type. I wish 2010 to be the year of a significant increase in software quality of Django apps. I’ll try to do my part.

Bookmark and Share

Tags: , , , , ,
Posted in Programming | 13 Comments »

« Older Entries