First let’s have a look at the supporting code:

from __future__ import with_statement
from fabric.api import *
from fabric.utils import warn


def apt_get(*packages):
    sudo('apt-get -y --no-upgrade install %s' % ' '.join(packages), shell=False)

Our apt_get fabric command issues apt_get install that answers yes to all questions and does not upgrade if the requested package is already installed. Now let’s take a look at the main command that installs mysql-server:

def install_mysql():
    with settings(hide('warnings', 'stderr'), warn_only=True):
        result = sudo('dpkg-query --show mysql-server')
    if result.failed is False:
        warn('MySQL is already installed')
        return
    mysql_password = prompt('Please enter MySQL root password:')
    sudo('echo "mysql-server-5.0 mysql-server/root_password password ' \
                              '%s" | debconf-set-selections' % mysql_password)
    sudo('echo "mysql-server-5.0 mysql-server/root_password_again password ' \
                              '%s" | debconf-set-selections' % mysql_password)
    apt_get('mysql-server')

First we want to make sure mysql-server is not already installed. For this reason we issue dpkg-query --show mysql-server. Note that if this command fails then mysql-server is not installed and we can proceed otherwise we want to return. We hide the ugly error messages by running our dpkg-query command within a settings context:

with settings(hide('warnings', 'stderr'), warn_only=True):
    result = sudo('dpkg-query --show mysql-server')
if result.failed is False:
    warn('MySQL is already installed')
    return

The rest of the code is pretty straightforward. We prompt to the user running fabric for the root password and seed its value twice into the debconf database.

sudo('echo "mysql-server-5.0 mysql-server/root_password password ' \
                            '%s" | debconf-set-selections' % mysql_password)
sudo('echo "mysql-server-5.0 mysql-server/root_password_again password ' \
                            '%s" | debconf-set-selections' % mysql_password)
apt_get('mysql-server')

Finally, having finished our little dance, we install mysql-server. I hope this helps some automated deployment believer out there.

Related posts:

1. Quest For Ultimate Development/Deployment Toolset: Fabric, Pip & Virtualenv
2. django-renderformplain
3. Developing Reusable Django Apps

By community support, I mean free software support provided by the user community. Every successful project has some form of communication; mailing lists, forums, IRC, wikis, etc. And the preferred language is almost always English. Simply because you can reach more people. People who can use your code. Who can test it, file bugs, send patches, etc. It is the logical thing to do. But in the spirit of freedom, I think nobody should be forced to use a certain (natural or programming) language for something they have created.

Real Communities

I think having local communities is a great idea. I don’t know, because we don’t have any in Türkiye². So I am just guessing they should be cultivating and fun at the same time. We do have many local pseudo-communities though. There is some activity, people come and go, some of them stay much longer than the others. But they never progress towards a community. I think this is because they make the fundamental mistake of localizing community support. They translate documentation intended for highly technical people and create new channels that no advanced user will bother to participate. In short, they attempt to sidestep the language barrier.

If want to be a programmer you need to know English. It is not optional. It is not a requirement of being a good programmer. You are at most an excellent script kiddie without proper English skills. Obviously, knowing perfect English doesn’t make you a great programmer instantly. It will increase the resources you can access dramatically, though. And most importantly it will give you the chance to know more about what you don’t know. Stay in your little world to play with your mates who know just as little as you do and you will never improve.

A Foolish Endeavour

Some people, who spend time with these pseudo-communities, know English well enough. They are probably acting with good intentions when they provide support in their native language. But they are actually wasting their time. Having been solved one specific problem doesn’t make the other person a better programmer. On the contrary they are giving local users another reason to avoid solving the real issue. The function of a community should be supporting each member’s continuous improvement. That doesn’t necessarily mean solving their technical problems for them³.

What is wrong with asking questions in a native language if on a local channel or face to face with local people? Nothing. There is absolutely nothing wrong with that. It would be silly to speak another language there. But if you really want to improve, you can tap into a greater source of information. It is your choice. Pseudo-communities will only take you so far. Because they are only generating more newbies, and not supporting newbies become experienced users. Do they produce anything but empty talk?

Why should a local community produce anything? What should the product be? I will probably write another post about this soon. But feel free to post your ideas and critics in the comments.

1: Obviously, it only makes sense if your native language is not English.

2: I would very much like to be proven wrong on this one. If you know any, please write a comment and don’t forget to include a website and date of the last meeting.

3: Also note that there is a difference between helping someone solve a problem and solving the problem. Spoonfeeding does more harm than good.

Related posts:

1. Nominate Qooxdoo for SourceForge Community Choice Awards
2. Psychic IRC Support In 10 Easy Steps
3. django-formfieldset

Free Software & Open Source Days of İstanbul Bilgi University and Linux & Free Software Festival of Linux Users Association are united under the name Free Software & Linux Days this year. If you have attended before, you will probably make no other plans for April 2-3.

If you have never been to this event, registration is free and can be done at the front desk. If you are remotely interested in free software or hackerdom you will want to be there. …and, of course, you are welcome.

I will be giving a Django presentation on Friday. Please come and say hello if you happen to be attending.

UPDATE: You can find the slides from presentation here. Slideshare’s importer failed to import the file I’ve uploaded properly. So please download and view the slides with Acrobat Reader.

Related posts:

1. django-formfieldset
2. What’s New in django-formfieldset 1.1
3. Freedays’09 Recap

New Example Project

There is a new and improved example project. It is designed to be some sort of documentation at the same time. When you run the example project and visit different pages you will see, for each examle, Python code, template code, text of rendered result and finally the result embedded.

If you have Pygments installed all the code will be nicely highlighted.

Fieldset & FieldsetMixin Improvements

Fieldset definitions are validated now. An exception will be raised if all of your fields are not included exactly once.

Template strings that are used by as_table, as_p and as_ul methods are now class attributes. You can simply override them instead of writing your own as_* method.

FieldsetMixin provides a fieldset_dict attribute. This dictionary has slugified fieldset names as keys and Fieldset instances as values. Your fieldset declarations can still be accessed from fielsets attribute.

Rendering Improvements

There are two rendering related improvements: individual fieldset rendering and renderform template filter.

Fieldset objects have as_table, as_p, as_ul methods just like forms. Errors from hidden fields are handled correctly, but you still need to call non_field_errors() to get the top level errors. Also it is template author’s responsibility to make sure all the fieldsets are rendered.

If as_* methods are not enough for you, with renderform filter you can render your forms or Fieldsets through a custom template. It works like this:

{{ form.fieldset_dict.mytitlerenderform:"myapp/mytitle_fieldset.html" }}

If you call it without an argument formfieldset/form.html template will be used.

Related posts:

1. django-formfieldset
2. django-renderformplain
3. Dynamic Translation Apps for Django

I agree with the suggestions from the original article. Yet, I would like to change the order a little bit; DVCS and then agile (with lowercase a) and then the rest. None of my points below are cool trends, in fact I can guarantee most of you will find them boring. But I think they are all important. OK, I hope you are all psyched now. Here we go:

1. Be Careful With The Buzz

Trends are cool. What could be wrong about following cutting edge stuff? We all want to be up to date, no? I think it’s good to follow the trends if you have the experience and the ability to filter the BS. I know a young developer who was constantly going back and forth between Rails/Ruby and Django/Python. I haven’t heard from him for a while, but he is probably still doing that same dance. Why? Because his considerations were solely based on buzz, not on simple requirements analysis or technical comparisons or personal experience.

2. Learn And Use An Old-Fashioned Modern Low-Level Scripting Language

To all the scripting people, like me, out there: you need to have an understanding of what’s happening under the hood. At the least to appreciate our high-level environments, at the most to become genuinely good programmers. Being a Python person myself, I think the best low-level language to be proficient for me is C. Many other high-level languages have C interfaces. So investing the time to learn C should pay off one way or the other.

3. Do Less Web Programming

Aren’t we doing a lot of web programming these days? Actually I think doing X development exclusively is bad for your programming muscles. Web programming, enterprise work or system scripting, it doesn’t matter. But web programming happens more than anything else. Maybe some of you have only been playing with it, but there are a huge number of us doing nothing but web programming. This is so sad; both in an individual level and for the community at large.

4. Learn How To Educate Yourself

What is a noob? Here is a definition and disambiguation (from newbie):

Newbs are those who are new to some task and are very beginner at it, possibly a little overconfident about it, but they are willing to learn and fix their errors to move out of that stage. n00bs, on the other hand, know little and have no will to learn any more. They expect people to do the work for them and then expect to get praised about it, and make up a unique species of their own.

Make an active effort not to be a noob. Learn how to ask smart questions, how to communicate others and seek help. Being polite is good but actually improving and being a valuable member of the community is much, much better.

5. Open Source Properly

It’s great to open source your project. But please do it properly. There are already too many unmaintained, undocumented projects out there that noone seem to care. Do you really have to add to that? As is argument doesn’t make much sense today. But if you really have to make an open source dead drop, please at least document the status of your project and your intentions clearly.

I wouldn’t be surprised if some you think they all are obvious. But if they are so obvious then why are they widely being ignored? Is it because they are under-retweeted, under-reddited and therefore not trendy.

Related posts:

1. django-formfieldset
2. What’s New in django-formfieldset 1.1
3. Why Not To Localize Community Support

I think apps¹ are one of the strong points of Django. A selling point if you like. There is a great ecosystem of apps, you can find an app for almost anything posssible with Python. And Python is kick-ass when it comes to library wealth. But there is another major advantage of apps when they’re done right; a sane code base. Here is a slide from the Django in the Real World presentation by Jacob Kaplan-Moss:

The fivefold path

• Do one thing, and do it well.
• Don’t be afraid of multiple apps.
• Write for flexibility.
• Build to distribute.
• Extend carefully.

I will focus on flexibility and interoperability of apps in this post. But before we proceed I would like to emphasize the first bullet point in the slide. Because the scope of your app plays a big role in its flexibility and interoperability. Apps should be small enough to easily understand and integrate (into a project). Many times I have moved away from an otherwise good app because of its many dependencies and/or excessive features. On the other hand apps should be big enough to allow for different configurations and allow extension without modifying their code. Do one thing, and do it well.

Take django-tagging for example; it’s 1.3 KLOC but it does tagging and nothing else. There are no dependencies other than Django, you can add tags to any model without modifying the model source, a tag can be associated with any type of model and tagging hides the gory details from you… In short; finding the right size is important. This is why tagging is the tagging app for Django.

Building For Reuse

General advice is “even project specific apps should be reusable“. Slapping the same app onto another project is not the only advantage. In fact it may not be possible if you are not in the habit of upgrading your whole project to recent versions of Django. The main advantage as I have said before is sanity. I prefer Django to other web frameworks/environments because it provides a civilized way of development. Let’s accept it; web programming is not a particularly interesting, exciting or intellectually rewarding field. You write the same piece of code over and over. And worst of all the challanges you face are actually a result of either the underlying system was designed by morons or you are trying to use it for something it’s not intended to be used. So it is only natural that web programmers feel they’re rusting. Django eases the pain. If you stick to certain conventions serenity will follow as well.

Naturally the framework does most of the work regarding app flexibility and interoperability. Take URLs for instance include('myapp.urls') and you are good to go. You don’t have to bind views one by one. Is it inflexible? Who said urls.py can only contain a hardcoded list of URLs. You can do anything that is possible with Python. You can generate different urlpatterns based on a setting for instance.

It is relatively easy and straightforward to reuse and extend forms and views (both function based and class based). Models are a little harder to get right though. You should always think of the most difficult situation which is you can’t touch either app’s code. Registration pattern of admin app provides a good solution here. You can register a third party model to another third party app in just a few lines.

You don’t need to write lots of code to get the flexibility and interoperability. Well designed apps make good use of settings.py for example. Why should the project developer wrap a view when a single line assignment would do the job? Supplying good templatetags and template snipplets (includes) is another way to make things easy for app consumers.

Signals provide a great way to propagate the events generated from your app. Even though they are one way², signals are extremely powerful. Any number of observers can connect to a signal and you can send a signal anywhere in your code. Literally. It is even possible your app suppying a signal and then another app sending it³.

There are many more ways to tame your app to be reusable. It all starts with your determination and discipline. Just like documentation, testing and maintaining a software generally. I will write more about reusable apps.

1: The word application is used both for a web application and a Django application. To avoid confusion I always use app to indicate the latter.

2: Signals don’t have return values. But you can use a callback AFAIK.

3: I can’t think of an example this would be useful, but still…

Related posts:

1. Developing Reusable Django Apps: Signals
2. Developing Reusable Django Apps: App Settings
3. Dynamic Translation Apps for Django

Django-multilingual

One of the few active projects I have reviewed is django-multilingual. To enable a model for translations you need to create a Translation inner-class and move translatable fields inside. Then a seperate model for those fields is created behind the scenes. Django-multilingual has admin integration and a multi-language flatpages app. Another nice thing about this app is that it’s using signals internally. The API is based on getter and setter methods. For instance you call get_<fieldname>(language_id=None) on your model to get the field value for the active translation. This app has no releases and less than satisfactory API documentation and has tests only on the example project.

Django-pluggable-model-i18n

Django-pluggable-model-i18n uses registration pattern of admin app. It creates an extra model for translated fields and stores translations of non-default languages in it. This app has no releases, no tests, no API documentation and it is clearly stated to be experimental. I would also like to note the last commit date is May 25, 2009.

Django-modeltranslation

Another app that implements registration pattern is django-modeltranslation. The advantage of registration pattern is you don’t have to modify the code of the 3rd party apps you want to translate dynamically. But the database schema still needs to be modified if you are using django-modeltranslation. Also there is one translations.py for the project. I think one translation definition file per app would be better (just like admin does). Django-modeltranslation has intuitive underscore-language_code API but it is somewhat inconsistent; when you read the unsuffixed field you read the currently active language, when you write you write the default language specified in settings.py. Also the modified model stores some redundant data. For instance if your DEFAULT_LANGUAGE is "ES" both <fieldname> and <fieldname>_es columns will store the same value. Django-modeltranslation has admin integration and a management command to update database schema. Most importantly this app is the only one which has both tests and a release. (Unfortunately tests require specific settings to run)

Transdb

Transdb takes a completely different approach to dynamic translation problem. It provides two new field types; TransCharField and TransTextField. Then it serializes all your translations within a single column for each field of those. This means no JOINs and no extra queries. Unfortunately transdb doesn’t implement underscore-language_code API, you need to use get_in_language() and set_in_language() methods. Transdb has default widgets that render one form field for each language. Last commit date is Nov 07, 2008 and there is a release.

Django-multilingual-model

This one is not actually an app but just one module with 33 42 lines of code. You need to define the model that holds translations manually. This introduces some code redundancy, since you also define which fields get translated in the original model. Django-multilingual-model doesn’t implement any translation API, so it’s rather verbose to do anything with it. There are no tests and no releases. I simply don’t recommend django-multilingual-model for anything serious.

Django-transmeta

Django-transmeta stores translations in extra columns it creates in the original field’s table similar to django-modeltranslation. But you enable translation assigning a metaclass for your model and then add a Meta attribute; this means you can’t make models in existing apps translatable without modifying their code. Django-transmeta implements underscore-language_code API, has admin integration and a management command to sync database when you add new languages or translatable fields. There are documentation and code examples but no tests or releases. Last commit date it Nov 24, 2009.

Comparison of Dynamic Translation Apps

Software development is making choices. Would you rather have a clean and stable schema with an extra translations model or avoid extra JOINs and denormalize translations onto your original model’s table? Both have advantages and disadvantages. But some choices are not based on trade-offs. Documentation, examples, tests and releases for instance. Also in my opinion underscore-language_code API is way better than any of the alternatives.

Django platform is a very powerful and intuitive one. Many people have moved in last year. This popularity affected app ecosystem as well. But unfortunately a significant number of those apps are half baked fire-and-forget type. I wish 2010 to be the year of a significant increase in software quality of Django apps. I’ll try to do my part.

Related posts:

1. Developing Reusable Django Apps
2. Developing Reusable Django Apps: Signals
3. Developing Reusable Django Apps: App Settings

An Example

After you copy renderformplain folder somewhere within your PYTHONPATH, add "renderformplain" to your INSTALLED_APPS setting to be able to run the example.

Let’s say you have a model File:

   1 class File(models.Model):
   2     name = models.CharField(max_length=100)
   3     path = models.CharField(max_length=250)
   4     size = models.IntegerField()
   5     last_modified = models.DateTimeField(default=datetime.datetime.now)
   6     created = models.DateTimeField(default=datetime.datetime.now)
   7     permissions = models.IntegerField()

And a form FileForm:

   1 class FileForm(forms.ModelForm):
   2     class Meta:
   3         model = File

Now assume you are using django.contrib.formtools.preview.FormPreview to review entered data before saving. In your formtools/preview.html template, instead of rendering the form as an HTML form, you can render it in plain text like this:

{% load renderformplain_tags %}

{% plainform form as plain_form %}

<h1>Preview your submission</h1>
{{ plain_form.as_table }}

<form action="" method="post">
  {{ form.as_hidden }}
  <input type="hidden" name="{{ stage_field }}" value="2" />
  <input type="hidden" name="{{ hash_field }}" value="{{ hash_value }}" />
  <p><input type="submit" value="Submit" /></p>
</form>

This will render the plain_form just like a normal form, except all fields will be replaced with read-only plain text.

Anyway. Try renderformplain and tell me what you think.

1: I will tag releases. Check out the repository for tags.

Related posts:

1. django-formfieldset
2. What’s New in django-formfieldset 1.1
3. Dynamic Translation Apps for Django

from django import forms
from formfieldset.forms import FieldsetMixin


class MyForm(forms.Form, FieldsetMixin):
    # Fields etc...

    fieldsets = ((u'Fieldset Title',
                  {'fields': ('foo', 'bar', 'baz'),
                   'description': u'This is the description for fieldset.'}),
                 (None,
                  {'fields': ('some_field',),
                   'description': u'This fieldset has no title.'}),
                 (u'Fieldset With No Description',
                  {'fields': ('some_other_field',)}))

Then you can render your form with fieldset enabled methods:

<form method="POST" action=""><table>{{ form.as_fieldset_table }}</table></form>

It is far from complete¹, but feel free to download and play with it.

1: Not released yet.

Related posts:

1. What’s New in django-formfieldset 1.1
2. django-renderformplain
3. Dynamic Translation Apps for Django

You can use the link below to vote yourself:

Related posts:

1. Why Not To Localize Community Support
2. Using Layouts In Qooxdoo – Part 1
3. Using Layouts In Qooxdoo – Index