Archive for the ‘Programming’ Category

Does Django Need An Overhaul?

Friday, September 26th, 2008

When I wrote Freedom is Power, I was pissed off because of the extra work of extracting all the info into the context for Django’s templating system doesn’t support enough logic. You have to create many little variables to be inserted only once in the template, while you could have just passed the model instance itself. Coding and maintenance becomes harder.

I have done some thinking over this but I am still not sure. If you’re doing both the views and templates yourself then it can be overkill. Just like static typing, or public/private/protected discrimination. On the other hand for design-heavy projects, where there is a seperate template designer who doesn’t know/care much about programming it makes sense. Still the possibility of having adequate presentation logic in templates is a good thing. It shouldn’t hurt anyone.

Last week (jinja dev) Armin Ronacher has posted an article named “Why Jinja is not Django and why Django should have a look at it“. It is a comprehensive comparison of internal workings of both templating engines. Inevitably it is a little biased, but I think each argument is valid and important. If you are into Django, it is a good read. There is also some speed comparison to back it up.

I am currently working on a project using Django. But I don’t touch the templating engine for this one. Everything served as JSON. :D

Bookmark and Share

Sum Of All Your Choices

Monday, August 25th, 2008

I used to have a discussion with an old friend about intellectually challenged and destitute of knowledge being happier and whether we should stop trying to become conscious and admit to nearest brainwashing center (TV?). I was the fool to advocate gnothi seauton (know thyself) and a meaningful life. Obviously he had a greater understanding of life then.

There are people (almost everybody)1 who are so oblivious about their miserable life, they are happy. They are not really happy but they are content. Judging by their actions, it is OK for them. Even if they continuously complain. Taking control of their life is simply not a viable option for them. When confronted they would either say they are already in control or give you an endless list of reasons why they can’t. So? Who cares?

I have recently finished reading In the Beginning was the Command Line. I didn’t like it too much in the beginning, but then as I read through I discovered it is a masterfully crafted article and the topic relevance for me is just an artifact. I suggest anyone, not just computer/internet enthusiasts, to read it. It is fun.

The article is about operating systems as interfaces. It has lots of colorful analogies and even some anectodes about interfaces and interaction. It explains that computing is all about building complexity. How UNIX builds complexity from simple components and GUI operating systems, such as Windows and Mac OS, hide complexity behind an interface. The distinction is that you control complexity in command line or you delegate control to your GUI. When you delegate anything you have much less control. You know this, right?

When you are working in the command line all the tools are available, but unmaterialized until you summon them. In a GUI environment all the tools are usually presented as choices and you simply pick one or the other. This reminds me Barry Schwartz’s TED talk; Paradox of Choice. The problem, sometimes, is not not having the right tools but the difficulty of choosing amongst too many2. It is the way of thinking suggested3 by your environment that makes the difference.

Before I say WYSIWYG is a joke, let me explain what it stands for; what you see is what you get. How honest and straightforward isn’t it? And it sometimes even deliver that promise. No, no, to be honest WYSIWYG implementations deliver most of the time, meaning the result is what you intended it to be. But what about those rare occurences of data4 loss or when your entire document corrupts when you try to add or change a small thing?

It drives me crazy when a application WYSIWYG starts to act like what you see is what you will but not necessarly what you want. At that point all the time saving and slick icons and so called wizards don’t mean anything to me. All I can think of is my computer doesn’t follow my orders, this is frustrating. I choose not to use WYSIWYG whenever possible. Explicit is better than implicit.

Before we move on let me be clear on one thing; being in control doesn’t necessarily mean having to do more. On the contrary ditching WYSIWYG should make to process more practical and efficient. You just have to use more brainpower and maybe to a little bit more planning than before. But once you have your setup in place, I assure you it will feel more like home.

I have two plugins loaded for this blog; google-analytics plugin and Markdown. Markdown is a text to HTML conversion tool. You write your document in a specific but intuitive5 format in plain text. All you need is a text editor. You get to work directly on the source, whereas you work on a representation of source in WYSIWYG. So it never blows up, no data loss can occur because you tried to change the formatting. I write my blog entires on Kate (my text editor) using Markdown format. I rarely need to check because it is easy to guess how it will look like when rendered.

So I can’t easily integrate charts6 and stuff, do fancy numbering or change my paper size7. Smart Ascii surely has its limitations. But do you really have to have those features? Just because the neighbour’s kid has?

Choosing plaintext doesn’t only help me keep my sanity, being a simple protocol is a great advantage. Let me give you some examples;

  • You can use regex search and replace on the source, meaning format together with the content. For example you can change all third level headings (“^### (.*)$“) into second level headings (“## \1“).
  • You can render into multiple formats easily. Or you can simply chain; using a HTML to PDF converter you can easily create PDF documents from the same source.
  • Plaintext can be diffed. You can see exactly what changed between two versions of a document. If you use a revision control system, with a little more effort, you get persistent undo (persistent between editing sessions).

This is not an exhaustive list of course. I just wrote what came to my mind at the moment. I use plaintext intensively and daily. It never breaks. You can open a plaintext document on any computer and you will probably be able to do so in the future as well. It is almost a trivial task to open a plaintext document, (very unlikely but) even if you can’t find an application you can write your own. Try Markdown, or any other Smart Ascii format. Once you get the initial (pyschological) barrier I hope you will like being in control.

1: Except of course whoever you are and all the people you know and care.

2: Have you ever worked on 3D Studio Max or Maya? If you are interested in interface design I would recommend you to spend some time on them. And then a little more with Blender.

3: Forced might be truer. And if you disagree here is Shelley’s blog.

4: Usually you lose formatting and not the content. But as far as I am concerned formatting is data as well.

5: Markdown and similar formats are called smart ascii. It is not necessarly ascii, I always use UTF8. But it is IMO smart in the sense that you can both read and edit the source document with the same ease.

6: Actually it is pretty easy. Markdown text can include HTML and using something like Google Charts API you can easily add charts.

7: If I were to produce a PDF or print out; I just pass my Markdown text through Dingus and paste the rendered text into OpenOffice Write. Doesn’t even take 1 minute.

Bookmark and Share

Freedom is Power

Monday, July 14th, 2008

If one sticks too rigidly to one’s principles, one would hardly see anybody. (Agatha Christie)

Web applications are thin client (your browser) applications you access through an intranet or the Internet. Due to initial design of WWW, to create rich applications you need to deal with quite a large number of protocols/languages/concepts. To name a few; you need to know about JavaScript, CSS, HTML (preferably XHTML) and (possibly) JSON to write an AJAX application. This is only client side, the front end.

To preserve sanity, a layered approach that divides and isolates different parts of the application as much as possible is usually preferred. MVC pattern, which stands for Model-View-Controller, is a popular method for layering. Model is the layer where your data storage and manipulation occur. View is the layer where the presentation of model occur. The controller layer is basically the glue between both, where your so-called business rules should be. MVC pattern allows us to change the user interaction or to adapt different back ends with minimal modifications.[1]

I would like to summarize how a web application using MVC pattern works. But before that I need to state that this is an overly simplified model I will present.

  1. When an HTTP request comes to our server a request object is created. This request object might have come modified according to our configuration before we have access to it. For example we might have wanted our request body to be decoded to unicode.
  2. The request object is passed to a view function, depending on our configuration again. URL patterns might be matched against view functions or, rarely, against HTTP method (GET, POST, etc) or an entirely different method can be used to determine which view function to call.
  3. The view function communicates with the model, adds, deletes or modifies the models as needed. Or it might just pull some data out and present it to user.
  4. We can generate the response from our view but, it is preferred to use another template layer. Because that way you can seperate application logic and presentation logic. In that case the view function returns the necessary information into a template object.
  5. The template object receives the data from the view and renders the final response (an HTML page, an XML document, etc).

We can pass data from view to template in a number of ways. We can use dictionaries (hashmaps) with arbitrary objects as values for example. When we return this dictionary from our view function the template objects picks it up and makes the data available by reference to its keys.

What we pass depends on the presentation logic in the template. Just to give you an idea, say we have a query page, where the user simply asks for a particular piece of data and we present it;

  • It will surely need a reference to that data. (Maybe a model instance if you are using an ORM)
  • It may present data conditionally, some parts might be available only to the administrators and invisible to normal users for example. So we would need to pass something to determine these conditions. I’ll come back to that something later.
  • It may have other elements (on the page) that need references to other data. (There might me a dynamically generated menu on certain pages for instance)

This means we would pass three kinds of references to a template; primary data, conditinal information and secondary data. I have mentioned that when we have templates we can seperate application logic (business rules?) and presentation logic. In (beautiful) web framework Django‘s documentation it is said;

…the template system is meant to express presentation, not program logic.

and

We see a template system as a tool that controls presentation and presentation-related logic — and that’s it. The template system shouldn’t support functionality that goes beyond this basic goal.

If we wanted to put everything in templates, we’d be using PHP. Been there, done that, wised up.

I read presentation as laying out output and program logic as inserting/querying/modifying/deleting. This takes us back to the something I would like to discuss. How do we pass information related to conditional rendering. What we pass depends greatly on the functionality of the template engine. Let me illustrate with an example;

We want to display a tip if;

  • A user who is registered for at least 2 weeks ago.
  • Who has less than 5 friends.
  • No other warning, such as a form validation error, is present on the page.

My intuition tells me I would need a reference to the list of errors and the (authenticated) user object in my template. I would like to have a reference to my user object because if I get anything less I might have to modify the view if the requirements change. So the registration date with the friend count would not do. After all we are deciding whether or not to show a tip, we are not doing inserting/querying/modifying/deleting I mentioned earlier.

Suppose we have user and error passed into our template, how do we use them; [2]

{% if user.registered.today().toordinal()
    -user.registered.toordinal() >= 14
    and user.friends.count < 5
    and not errors %}
    <div class="tip"> ... </div>
{% endif %}

Of course this does not work in Django. Django template engine’s if doesn’t have greater-than-or-equal operator, less-than operator and can not combine with truth testing (Instead has a seperate ifequal statement for truth testing). There are two workarounds;

  1. We calculate conditions at the view and pass them as simple boolean values.

    {% if user_registered_for_at_least_two_weeks
        and user_has_less_than_five_friends
        and no_errors %}
        <div class="tip"> ... </div>
    {% endif %}
    
  2. We pass user reference and add checking functions to the models.

    {% if user.registeredAtLeastTwoWeeksAgo
        and user.hasLessThan5Friends %}
        {% ifequal errors None %}
            <div class="tip"> ... </div>
        {% endif %}
    {% endif %}
    

The first one is actually not a solution, since we calculate the conditions in the view the presentation code is spanning both layers. The second workaround allows us to decouple the template and the view but adding more and more functions on models for such simple operations is just displacing the problem.

As far as I understand Django template engine has very simple control flow functionality for a reason. To ensure logic stays in the view; limit functionality in the template (to truth testing and equality testing only). But eventually you want a somewhat complex template and either the view or the model gets cluttered.

The first sample, the one with the complex if, is jinja code. It’s conditional staments are almost as powerful as Python. This allows bad programming of course. You can do almost everything you are supposed to do in the view. But you if you follow decoupling principle you can have your code working, clean and maintanable.

I think Django is a special project (as an open source project, like Blender, like Inkscape in this sense). If we look carefully there are many lessons to be learned. Both in the API level and in the source code itself. I try to learn as much as possible from Django. But on my personal projects I would like to take full advantage of the expressive power of jinja (and sqlalchemy similarly). I try to assemble best components available, but I also do my best to follow best practises (learned from Django or other sources).

If you are considering Python for web development, I would say learn Django first. And then go ahead and study other frameworks and libraries. Python is extremely powerful and suits web development well. Once you found the setup (framework or whatever) that works best for you, I am sure you will enjoy it.


1: There are different interpretations of MVC model. What happens in each layer might slightly change depending on the application type and of course implementors.

2: I know these line breaks look stupid. But it blows the page away otherwise.

Bookmark and Share