Posts Tagged ‘python’

Working with files in Django – Part 3

Wednesday, November 30th, 2011

First part of this article is here.

How to add STATIC files

You will most likely add STATIC files to your source code repository. As they are likely to be hardcoded in your code and templates it is a good idea to keep those and your STATIC files in sync.

Your STATIC files are collected (found and copied or symlinked) with the help of finders. This collection process might be a little confusing for you. Basically you don’t need to worry about where your files are copied, you just need to maintain the files in the locations I mention below. Collection process consolidates all your STATIC files for you, and it does it automatically.

If you are developing a reusable app place your STATIC files under the static directory. Your app directory should look like this:

$ ls -1
models.py
static/
templates/
views.py

If you are developing a project absolute filesystem paths listed in your STATICFILES_DIRS setting will be collected:

>>> from django.conf import settings
>>> settings.STATICFILES_DIRS
('/opt/myproject/src/project/static',)

One important thing to note, as I mentioned in the first part, is to set your STATIC_ROOT outside of your project directory. This is the location where all the STATIC files found will be consolidated. Making this directory independent from your project installation enabled its reuse.

How to upload MEDIA files

MEDIA files are typically uploaded by users when the project is online and a reference is stored in a model field. While this is true most of the time, MEDIA files can be generated by code and/or a reference can be provided in some other way that doesn’t involve models. But these edge cases are out of the scope of this post.

The easiest way to allow users upload their files is to use a FileField or ImageField on a model and derive a form from it:

from django.db import models
from django.forms.models import modelform_factory


class MediaModel(models.Model):
    media_file = models.FileField(upload_to='user_media')


MediaForm = modelform_factory(MediaModel)

You can then use this form to provide upload functionality:

def media_create(request):
    if request.method == "POST":
        form = MediaForm(request.POST, request.FILES)
        if form.is_valid():
            form.save()
            return HttpResponseRedirect(reverse('media-list'))
    else:
        form = MediaForm()
    return render_to_response('usermedia/create/html', {'form': form})

Of course it is better to use a generic CreateView now we have them with Django 1.3. I wanted to emphasize one point and avoid the complexities of a class based view; you must pass request.FILES to the form’s constructor. Actually this is a good practice whether or not your form has an file field.

As in the case with STATIC files, MEDIA_ROOT should be in a location seperate from your project files. If you delete your project directory you would also lose MEDIA files otherwise.

Conclusion

This concludes Working with files in Django. I hope these posts are helpful to you. Once you are comfortable working with files I strongly recommend you to take a look at Django Compressor

If you enjoyed this post please tell me a little bit about yourself.

Bookmark and Share

Working with files in Django – Part 2

Saturday, November 26th, 2011

First part of this article is here.

How to setup file serving for development server

As I mentioned earlier, it is best to serve files on a fast HTTP server. But this setup is overkill for development environments, you can safely let Django handle your files. The snippet below relies on the assumption, behind the scenes, that DEBUG is always True in development environment and it is always False in production environment. It should be fine for the purposes of this post:

from django.conf.urls.static import static
from django.contrib.staticfiles.urls import staticfiles_urlpatterns

urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
urlpatterns += staticfiles_urlpatterns()

It is quite straightforward. Nothing gets added to urlpatterns unless DEBUG is True and prefix (MEDIA_URL and STATIC_URL) is not a fully qualified URL.

How to reference STATIC files in templates

There are two ways to reference any file. You can use a file’s location in your file storage (your filesystem for example) for internal use. Or you can build a URL for the file so that clients can access it. This post will cover the latter, as the former is a simple matter of applying os.path.join() to STATIC_ROOT and the particular file’s path.

To build a URL in your templates you can simply concetenate STATIC_URL with the file’s relative location to it:

<img src="{{ STATIC_URL }}myapp/img/logo.png" />

However, to have STATIC_URL available within your template context you need to make sure of two things:

  • The template needs to be rendered with a RequestContext.
  • django.core.context_processors.static needs to be included in TEMPLATE_CONTEXT_PROCESSORS setting.

I can’t think of a reason why, but if for some reason you are not using RequestContext, you can provide STATIC_URL using getstaticprefix tag:

{% load static %}
{% get_static_prefix as STATIC_URL %}

<img src="{{ STATIC_URL }}myapp/img/logo.png" />

How to reference MEDIA files in templates

Dealing with MEDIA files is much simpler. The FieldFile object returned by ImageField and FileField has path and url properties and you don’t need to do concetenation yourself:

<img src="{{ some_model.some_image_field.url }}" />

Next part deals with adding files to your project.

If you enjoyed this post please tell me a little bit about yourself.

Bookmark and Share

Working with files in Django – Part 1

Monday, November 21st, 2011

In this post what I mean by file is any content that is not dynamically generated. There are basically two types of files a web application deals with:

  • Files that are hard coded in templates or in code. We will call them STATIC files.
  • Files that are referenced in the code but only known in the run time. We will call them MEDIA files.

Now of course all these files are static and most of them can be classified as media, but I chose those terms because that more or less how they are referred to in Django documentation.

Since files are stored on disk or some other storage backend it is best to serve them on a fast HTTP server and let Django handle only dynamic content.

How to configure STATIC/MEDIA related settings

Let’s look at some code first and we can comment on it afterwards. I use the following setup in my projects:

import os

_PATH = os.path.abspath(os.path.dirname(__file__))

MEDIA_ROOT = os.path.join(_PATH, 'files', 'media')
MEDIA_URL = '/media/'

STATIC_ROOT = os.path.join(_PATH, 'files', 'static')
STATIC_URL = '/static/'
STATICFILES_DIRS = (
    os.path.join(_PATH, 'static'),
)
STATICFILES_FINDERS = (
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
)

ADMIN_MEDIA_PREFIX = '/static/admin/'

_PATH should be obvious. But the way I use it is a little tricky here. Basically I am using a files directory that contains media, for MEDIA files, and static, for STATIC files, directories. This seperation is essential. You will pull your hair if you don’t seperate those two from the beginning. This will allow you to reuse them easily if you deploy a new version to a different location.

But why is files under the project directory? Because we need to check-in STATIC files? NO! It is just a convenience for the development environments. I strongly suggest overriding MEDIA_ROOT and STATIC_ROOT in your deployment settings and basically move the files directory outside of the project path. STATIC files that we check-in resides in <_PATH>/static. django.contrib.staticfiles app collects your STATIC files from your project specific directories and within your apps and copies everything in the right places inside STATIC_ROOT.

To sum up the points above:

  • Check-in STATIC files in a folder within your project directory (see STATICFILES_DIRS & FileSystemFinder). If you are developing an app AppDirectoriesFinder will collect static files within the static directory in your app directory.
  • Seperate the directories, where your MEDIA files are saved and where your STATIC files are collected (see STATIC_ROOT & MEDIA_ROOT).
  • Keep your STATIC_ROOT and MEDIA_ROOT outside of your project directory in production environment.

One more thing to note is STATIC_URL and MEDIA_URL should both have a trailing slash. This will save you a lot of trouble later.

Next part deals with serving files in development environment and referencing them in code and templates.

If you enjoyed this post please tell me a little bit about yourself.

Bookmark and Share

My PyCon APAC 2011 Presentation: Optimizing Media Performance with django_compressor

Tuesday, June 28th, 2011

I have given a presentation about django_compressor at PyCon APAC 2011. Slides are below for everyone to see:

Bookmark and Share

Drawing Gradients with PyGame

Saturday, November 20th, 2010

I have been learning myself some PyGame on my free time. Best way to learn new technologies, I believe, is to develop small project and doing a lot of reading. Everybody seems to agree on the former but most fail to put effort doing the latter.

I needed to draw gradients for the background of my little learning project. First I tried the naive approach: iterating over each pixel and setting the correct value. This takes forever as you could have guessed. I continued my research to find out the close connection between PyGame and NumPy. Yes, fast array operations. My gradient generating code using linspace and tile is below:

def generate_gradient(fromcolor, tocolor, height, width):
    channels = []
    for channel in range(3):
        fromvalue, tovalue = fromcolor[channel], tocolor[channel]
        channels.append(numpy.tile(numpy.linspace(fromvalue, tovalue, width),
                                                                  [height, 1]))
    return numpy.dstack(channels)

I create three 1D arrays for each channel and then tile them to get a 2D array. Then I stack those 2D arrays and get combined (r, g, b) values. Each 1D array contain values linearly sampled between given maximum and minimum values by linspace. I can then blip this array on a surface like this:

gradient = generate_gradient((63, 95, 127), (195, 195, 255), 1024, 600)
pygame.surfarray.blit_array(some_surface, pygame.surfarray.map_array(some_surface, gradient))

This works quite well. But I was curious whether there is a better method, so I did some googling. First I stumbled upon James Tauber’s gradient code. Arrays! I was a bit ashamed not remembering standard library had array module. James Tauber’s code builds an array with the correct values and then writes the image as a PNG file. It also has the ability to generate multi-gradients.

Then I found gradients package for PyGame. This package does much more than linear gradients. It creates a PyGame surface of 1 pixel thickness, fills it with the gradient values and then resizes it.

I made a small test to see which method runs faster. I had to strip file and compression related code from James Tauber’s gradient module. I run a simple method tha just generates a gradient image/array in memory and returns it 10 time, below is the average times:

  • 320×240:
    • numpy: 0.01469659805
    • array: 3.6310561180
    • pygame: 0.0069756984711
  • 800×600:
    • numpy: 0.1323119163
    • array: 19.9719331265
    • pygame: 0.0192141056061

Here numpy is my method above, array is the stripped version of James Tauber’s code and pygame is the gradiends package. If my benchmarking method doesn’t have dramatic errors fastest way to generate linear gradients is to fill a strip with correct values and then to resize it.

Now I’ll go back to work on my learning project.

Bookmark and Share