When I decided to give this software a fair chance to succeed on its very own domain the only question in my head was; will it pass the test of users? It doesn’t really matter what you have intended the users do with your application. What matters, first, is what they think they’d like to do with it and then whether or not they actually use it.

So I tried and I failed. Two main reasons of this failure are; technical deficiencies and the special way of interaction coffee reading is. Technical deficiencies is the easy one. I couldn’t devote enough time for telvee, especially lately. As a result it doesn’t even have basic stuff like e-mail changing or account deletion. This is 100% my fault. The second reason however is more complicated and there was not much I could do about it. Except one thing I will tell you at the end of this post.

Telvee didn’t pass the user’s test mainly because coffee reading is somewhat private. It’s more of a 1-to-1 communication, while all social applications¹ are designed for 1-to-many communication. This was disastrous not only because of the lack of viral growth but also because of the reluctance of users to interact with other users.

An example image of coffee remains telvee generates

I would like to thank everybody who participated and I hope you had some fun playing with it. Telvee domain will soon redirect to this post and I will probably not renew it next time.

I won’t be starting a new experiment soon. I will spend most of my time on my work. Hopefully I will be spending a little more time on free software projects. By the way I would like to note that Telvee has spawned a couple of Django apps: django-inviting & django-simple-friends.

Oh, and the thing I could do better about user experience was to be more agile. I should have either fixed the problem quickly or failed fast. I have no regrets though. This was a unique experience and I have learned a lot.

1: Yes, even the e-mail system and dating sites are designed for 1-to-many communication primarily in mind.

Related posts:

1. New Project: Telvee
2. Psychic IRC Support In 10 Easy Steps
3. Dynamic Translation Apps for Django

Creating A Base Debian System

We will create a Qemu machine and install Debian Lenny on it first:

# Download Debian image
wget http://debian.osuosl.org/debian-cdimage/current/i386/iso-cd/debian-504-i386-businesscard.iso

# Create base VM image
qemu-img create -f qcow2 debian.qcow2 2G

Our disk image will have a 2 Gigabyte size limit. You can pick a different size if you need.

Now we need to power on our VM and install Debian:

# Install Debian
qemu -enable-kvm -k tr -cdrom debian-504-i386-businesscard.iso -hda debian.qcow2 -boot d

You don’t need to allocate a large swap disk, 128MB should do just fine for a file/web server. Also I wouldn’t bother creating a seperate partition for /home/.

Next let’s log in as the user (www here) we have created to make final changes:

# logged in as user
dpkg-reconfigure console-data
aptitude install ssh sudo
echo "www ALL=(ALL) ALL" >> /etc/sudoers

At this point you might want to take a backup of debian.qcow2. (Even though we will open it only read-only from now on)

Creating The Actual VM

To save time and space we will use copy-on-write disks and re-use debian.qcow2.

# Create the actual VM's disk
qemu-img create -f qcow2 -o backing_file=debian.qcow2 actual.qcow2

Actually we are done. You can log in to your VM using the following command and start installing/configuring/running:

qemu -enable-kvm -k tr -hda actual.qcow2 -net user -net nic \
                                         -redir tcp:5022::22 \
                                         -redir tcp:9080::80

A few things to note about the command above:

• -enable-kvm is meaningful only if you have kvm kernel module installed. It improves performance a great deal, so it’s highly recommended.
• You probably need to change -k tr according to your keyboard’s layout.
• We are setting up two TCP redirections. 22 is for SSH and 80 is for HTTP. You can add more ports if you need.

Finally, I suggest you to prefer SSHing your VM instead of logging in directly:

# SSH into the VM
ssh -p 5022 www@localhost

I hope some of you find this useful.

Related posts:

1. GALFTHW Style Tutorial On Python Coroutines
2. Freedays’09 Videos
3. Psychic IRC Support In 10 Easy Steps

First let’s have a look at the supporting code:

from __future__ import with_statement
from fabric.api import *
from fabric.utils import warn


def apt_get(*packages):
    sudo('apt-get -y --no-upgrade install %s' % ' '.join(packages), shell=False)

Our apt_get fabric command issues apt_get install that answers yes to all questions and does not upgrade if the requested package is already installed. Now let’s take a look at the main command that installs mysql-server:

def install_mysql():
    with settings(hide('warnings', 'stderr'), warn_only=True):
        result = sudo('dpkg-query --show mysql-server')
    if result.failed is False:
        warn('MySQL is already installed')
        return
    mysql_password = prompt('Please enter MySQL root password:')
    sudo('echo "mysql-server-5.0 mysql-server/root_password password ' \
                              '%s" | debconf-set-selections' % mysql_password)
    sudo('echo "mysql-server-5.0 mysql-server/root_password_again password ' \
                              '%s" | debconf-set-selections' % mysql_password)
    apt_get('mysql-server')

First we want to make sure mysql-server is not already installed. For this reason we issue dpkg-query --show mysql-server. Note that if this command fails then mysql-server is not installed and we can proceed otherwise we want to return. We hide the ugly error messages by running our dpkg-query command within a settings context:

with settings(hide('warnings', 'stderr'), warn_only=True):
    result = sudo('dpkg-query --show mysql-server')
if result.failed is False:
    warn('MySQL is already installed')
    return

The rest of the code is pretty straightforward. We prompt to the user running fabric for the root password and seed its value twice into the debconf database.

sudo('echo "mysql-server-5.0 mysql-server/root_password password ' \
                            '%s" | debconf-set-selections' % mysql_password)
sudo('echo "mysql-server-5.0 mysql-server/root_password_again password ' \
                            '%s" | debconf-set-selections' % mysql_password)
apt_get('mysql-server')

Finally, having finished our little dance, we install mysql-server. I hope this helps some automated deployment believer out there.

Related posts:

1. Quest For Ultimate Development/Deployment Toolset: Fabric, Pip & Virtualenv
2. django-renderformplain
3. Dynamic Translation Apps for Django

Let me reiterate the last sentence. Having a Django-based blog just because Django is fashinable is a little dumb in my opinion. If Django-based X blogging engine suits you better than anything else, use it. Why not? But my personal choice of blogging engine is WordPress¹. The value of a Django blogging app, for me, is in adding a blog to a Django project. And different projects might have different requirements. So my idea of a Django blogging app is one that is highly configurable and highly extendable.

On the other hand I don’t need the convenience of clicking a checkbox on a polished UI. I can write a function. Or I don’t necessarily need it to, say, provide a navigation menu. There are apps that do that. Even if there wasn’t it shouldn’t be the blog app’s job. So I am not looking for an instant-blog. I have a Django app in my mind, nothing more.

What Should Be Left Out

Basically any feature that can be provided by another reusable app should be left out. Why should we re-implement something that is already done… and reviewed by others… and tested. Of course this doesn’t necessarily mean providing no convenience functions.

• No admin. Because we already have one.
• No theming. For the love of Flying Spaghetti Monster, you don’t need any theming other than what django.template offers. Pre-built themes are for turn-key solutions.
• No comments or contact forms. (See django.contrib.comments and django-contact-form)
• No official markup format (or formats). This can be handled in the templates without difficulty. But, maybe, pluggable content filters is a good idea. I haven’t made up my mind on this one entirely. It won’t use any markup format by default, that is for sure.

What Should Be Included

Remember, every project has a different set of needed features for its blog. Some need catagories, some need tags and some others need both. But it would end up as a disaster if we implemented each one of those features into a single app. Instead I think it should consist of many small apps that work together. But I wouldn’t want to end up having huge spaghetti of apps that all depend on one another, like Pinax does. A minimal amount of core apps² and then everything else should be optional. By optional I mean you don’t have to install packages you won’t need.

I think the components (apps) should be activated via adding to INSTALLED_APPS and configured with settings. I can’t think of any parameter that needs to be changed dynamically, so why not use the established way of doing configuration in Django.

Two must have features for such a blogging app are previews and scheduled publishing. It is possible that you sometimes write a post quickly and publish it immediately. But I suppose nobody will say they don’t care about these two features.

Built-in feeds and sitemaps are also nice to have.

Multiple instances of this blogging app running on the same project? À la admin. I can’t make my mind on this one. Sure it would be a nice feature. But it could complicate the code. Peehaps too much for a not so common case.

What do you think about the general idea? Are there any other must-have features? Would you be willing to learn a new app when you are already comfortable with another blogging app?

1: Even though it’s written in the abomination called PHP. But since there are plugins for everything I don’t have to touch the code.

2: One sounds like a good number, if possible.

Related posts:

1. Django: Where Should My App Live
2. Developing Reusable Django Apps: App Settings
3. Developing Reusable Django Apps

By community support, I mean free software support provided by the user community. Every successful project has some form of communication; mailing lists, forums, IRC, wikis, etc. And the preferred language is almost always English. Simply because you can reach more people. People who can use your code. Who can test it, file bugs, send patches, etc. It is the logical thing to do. But in the spirit of freedom, I think nobody should be forced to use a certain (natural or programming) language for something they have created.

Real Communities

I think having local communities is a great idea. I don’t know, because we don’t have any in Türkiye². So I am just guessing they should be cultivating and fun at the same time. We do have many local pseudo-communities though. There is some activity, people come and go, some of them stay much longer than the others. But they never progress towards a community. I think this is because they make the fundamental mistake of localizing community support. They translate documentation intended for highly technical people and create new channels that no advanced user will bother to participate. In short, they attempt to sidestep the language barrier.

If want to be a programmer you need to know English. It is not optional. It is not a requirement of being a good programmer. You are at most an excellent script kiddie without proper English skills. Obviously, knowing perfect English doesn’t make you a great programmer instantly. It will increase the resources you can access dramatically, though. And most importantly it will give you the chance to know more about what you don’t know. Stay in your little world to play with your mates who know just as little as you do and you will never improve.

A Foolish Endeavour

Some people, who spend time with these pseudo-communities, know English well enough. They are probably acting with good intentions when they provide support in their native language. But they are actually wasting their time. Having been solved one specific problem doesn’t make the other person a better programmer. On the contrary they are giving local users another reason to avoid solving the real issue. The function of a community should be supporting each member’s continuous improvement. That doesn’t necessarily mean solving their technical problems for them³.

What is wrong with asking questions in a native language if on a local channel or face to face with local people? Nothing. There is absolutely nothing wrong with that. It would be silly to speak another language there. But if you really want to improve, you can tap into a greater source of information. It is your choice. Pseudo-communities will only take you so far. Because they are only generating more newbies, and not supporting newbies become experienced users. Do they produce anything but empty talk?

Why should a local community produce anything? What should the product be? I will probably write another post about this soon. But feel free to post your ideas and critics in the comments.

1: Obviously, it only makes sense if your native language is not English.

2: I would very much like to be proven wrong on this one. If you know any, please write a comment and don’t forget to include a website and date of the last meeting.

3: Also note that there is a difference between helping someone solve a problem and solving the problem. Spoonfeeding does more harm than good.

Related posts:

1. Nominate Qooxdoo for SourceForge Community Choice Awards
2. Psychic IRC Support In 10 Easy Steps
3. Freedays’09 Videos

Free Software & Open Source Days of İstanbul Bilgi University and Linux & Free Software Festival of Linux Users Association are united under the name Free Software & Linux Days this year. If you have attended before, you will probably make no other plans for April 2-3.

If you have never been to this event, registration is free and can be done at the front desk. If you are remotely interested in free software or hackerdom you will want to be there. …and, of course, you are welcome.

I will be giving a Django presentation on Friday. Please come and say hello if you happen to be attending.

UPDATE: You can find the slides from presentation here. Slideshare’s importer failed to import the file I’ve uploaded properly. So please download and view the slides with Acrobat Reader.

Related posts:

1. django-formfieldset
2. What’s New in django-formfieldset 1.1
3. Freedays’09 Seminar Program

New Example Project

There is a new and improved example project. It is designed to be some sort of documentation at the same time. When you run the example project and visit different pages you will see, for each examle, Python code, template code, text of rendered result and finally the result embedded.

If you have Pygments installed all the code will be nicely highlighted.

Fieldset & FieldsetMixin Improvements

Fieldset definitions are validated now. An exception will be raised if all of your fields are not included exactly once.

Template strings that are used by as_table, as_p and as_ul methods are now class attributes. You can simply override them instead of writing your own as_* method.

FieldsetMixin provides a fieldset_dict attribute. This dictionary has slugified fieldset names as keys and Fieldset instances as values. Your fieldset declarations can still be accessed from fielsets attribute.

Rendering Improvements

There are two rendering related improvements: individual fieldset rendering and renderform template filter.

Fieldset objects have as_table, as_p, as_ul methods just like forms. Errors from hidden fields are handled correctly, but you still need to call non_field_errors() to get the top level errors. Also it is template author’s responsibility to make sure all the fieldsets are rendered.

If as_* methods are not enough for you, with renderform filter you can render your forms or Fieldsets through a custom template. It works like this:

{{ form.fieldset_dict.mytitlerenderform:"myapp/mytitle_fieldset.html" }}

If you call it without an argument formfieldset/form.html template will be used.

Related posts:

1. django-formfieldset
2. django-renderformplain
3. Dynamic Translation Apps for Django

It’s good to see CherryPy project is pretty much alive. It’s a web server and a web framework and more. Check it out if you haven’t!

Related posts:

1. Freedays’09 Videos
2. Nominate Qooxdoo for SourceForge Community Choice Awards
3. Psychic IRC Support In 10 Easy Steps

Taking advantage of signals doesn’t necessarily mean providing no sane defaults. You can send signals and provide default event handling. Here is a couple of ideas how this can be done:

• Your app can check if there are any listeners and connect the default handlers if there is none.
• You can ship an auxiliary app that connects default handlers when added to INSTALLED_APPS.

I personally prefer the second approach since it’s simpler and more explicit. I’m sure there are other ways to implement default handlers for signals.

Dispatch_uid

Don’t forget to assign a unique dispatch_uid for each connect() call. Otherwise your handler can get connected twice. I would also suggest you to use both module path and your handler function’s name in your dispatch_uid:

"%s.%s" % (os.path.splitext(__file__)[0].replace(os.sep, '.')[1:],
           handler_name)

Now I should take my own advice and replace hardcoded User.message_set.create()s with signals in django-simple-friends.

Related posts:

1. Developing Reusable Django Apps
2. Developing Reusable Django Apps: App Settings
3. Dynamic Translation Apps for Django

I follow (and read with great interest) Steve Souders’s blog High Performance Web Sites. I must admit I was sceptical about it at first; spriting images, individual different loading behaviours of browsers… I thought they were premature optimizations. But I realized I was wrong as I continued to read. Steve Souders is an expert on high performance web sites and what he preaches are realistic techniques, backed by test results most of the time. If you are not following, I suggest you add it to your RSS reader.

Optimization Techniques

We can explore the techniques in two main categories:

1. Techniques to reduce the data to be transferred.
   • Minifying: Minification is removing comments and unneeded whitespace in CSS and JavaScript files. Compilers like YUI Compressor and Closure modifies JavaScript code to compress even further, without any changes to the functionality.
   • Gzipping: Web browsers accept gzip encoded content for a long time. I have just compressed a 13 KB text file down to 4 KB. A two thirds compression ratio is not bad at all.
2. Techniques to reduce the number of HTTP requests.
   • Combining: CSS and JavaScript files can be combined together into a single file and therefore a single HTTP request. Gzip may be more efficient on these files. In a similar way background images can be merged into a sprite and then reconstructed again using their coordinates on that sprite.
   • Data URI’s: Images (or other file types) can be embedded into CSS (or JavaScript or HTML) using data: URI‘s. Extra HTTP requests for those resources can be avoided this way.

You might think it’s fine to perform all these optimizations, but what happens when I want to make some changes to my combined, minified JavaScript file? Instead of applying these techniques blindly, it’s best to follow a sensible plan for implementing these optimizations:

• First of all everything that can be automated should be automated. Regarding the example above script files should stay uncombined and uncompressed in the development environment and optimizations should be applied when the application is published. Taking it a step further we can have the application detect changes in those files and update optimized versions automatically. (django_compressor works this way)
• I was worried that spriting would complicate managing the design. But I have seen, on the contrary; if images each sprite contain are choosen carefully it makes the process easier. Start combining images that belong to the same design element. Avoid complex arrangements, stick with horizontal or vertical stacking as much as possible. Don’t forget to leave transparent spaces between items and sprite border. Try to combine images that are loaded on the same page, avoid loading a sprite for only half of it’s elements. Don’t force yourself to combine all images, if you follow the guidelines I have mentione they won’t.
• When performing optimizations don’t forget to use easily available tools. You can use YSlow for general analysis, SpriteMe for image combining tips, CompressorRater to compare different compilers’ performance on your scripts. I would like to note that Steve Souders is the developer of first two.

Telvee Results

I didn’t think about performance at all when I started developing telvee. Too many CSS files and too many images were being loaded. Here is what it looked like before optimizations:

Then I have installed and configured django_compressor. I used YUI Compressor for both JavaScript and CSS. I have created sprites and modified CSS files manually¹. Then I deployed these changes and measured again:

In the load column of Homepage, the number in parens is the actual load. But the design of homepage is changed with this upgrade and a new 90 KB image is being loaded now. So I have accepted 70 KB in my calculations. The result of optimizations are as follows:

Django_compressor and Data URI’s

Django_compressor, developed by Christian Metts, helps you to apply optmizations I have mentioned above easily to your Django projects. You can see my fork here where I have merged some other branches and added a little bit of code myself.

Using compressor.filters.datauri.CssDataUriFilter in data-uri branch of this repository, you can embed linked files within your CSS files. It will only embed files less than or equal to 1024 Bytes (1 KB) by default. You can change this limit by setting COMPRESS_DATA_URI_MIN_SIZE in your settings.py.

There are a couple of things to pay attention when you convert your references to data: URIs. Firstly file contents are base64 encoded which means approximately one third increase in size. It’s up to you to balance between increased bandwidth and reduced request counts². Another thing to watch for is multiple references to the same file will end up embedding the same data many times. The solution to this problem is to reduce all references to one³ but this might break your CSS arrangement strategy.

Please test django_compressor’s data: URI support and tell me what you think. If you haven’t applied optimizations I mentioned above, you should. Thanks to django_compressor they are quite easy to implement on Django projects.

1: I would like to add automatic sprite building/linking support to django_compressor sometime.

2: With Today’s modern connections 1~2 KB increase is a good price for 1 less HTTP request..

3: http://meiert.com/en/blog/20090401/why-css-needs-no-variables/

Related posts:

1. Sad State of Web Development Industry in Türkiye
2. Django Fixtures
3. Dynamic Translation Apps for Django