I stumbled upon this post following my frustration with setuptools after virtualenvising telvee repository. I have played with buildout to create repeatable deployments before. Don’t get me wrong, buildout is superb. But I have a few minor issues with it:

• Buildout can create an interpreter but the isolation from the host system is not a good as virtualenv. (Please correct me if I’m wrong)
• Buildout is a dependency¹ whereas virtualenv is a system-wide tool.
• It is, naturally, more work to set it up than a couple of ad-hoc bash scripts.

None of these issues make buildout less awesome, I am just not comfortable enough with it. I am considering other alternatives and their strengths and weaknesses.

Virtualenv

I am quite comfortable with virtualenv. It creates an isolated Python environment for you. For instance the following command will create an environment inside test directory that doesn’t have access to packages installed system-wide:

~/$ virtualenv --no-site-packages test

That is of course when you activate the virtual environment with:

~/$ cd test
~/test/$ source bin/activate
(test)~/test/$

Note the (test) prefix to your prompt. Any packages you install within this environment will be installed only for itself and won’t be available system-wide. If you have omitted --no-site-packages argument you could have access to globally installed packages too. When you are done with this virtual environment you can issue deactivate command to return to your normal shell. That’s basically what virtualenv does.

You can script virtualenv to a certain extend:

#!/usr/bin/env bash

virtualenv $1
cd $1
source bin/activate

echo $PATH
echo $PYTHONPATH

Or you can create bootstrap scripts with virtualenv. Virtualenv doesn’t provide a clean and powerful enough API here, just a callback and two methods to modify commandline arguments.

Virtualenv is great at what it does. But I think setuptools as a package installer (and it’s just an installer, not a manager) cripples virtualenv. Good news is newer versions have a commandline argument --distribute that appereantly substitutes distribute for setuptools.

Pip

My experiments with pip went just fine. Except I bumped into this problem. Globally installed pip was seeing system-wide packages when used on a virtual environment created with --no-site-packages. Again, good news is it is fixed in the trunk.

I will play with pip more once figure out how to best integrate the these applications.

Fabric

I see fabric as the glue that binds everything together. I have known about it long before, but I never had the chance to experiment. I have read most of the documentation² and played with it a little. Basically the following script is an attempt to create a virtual environment and install pip and django on it, much like the shell script above:

from __future__ import with_statement
import os
from fabric.api import *
from fabric import context_managers


def _get_virtualenv_location():
    location = prompt('New location: ', default='../test')
    env.envdir, env.envname = os.path.split(os.path.abspath(location))
    env.envpath = os.path.join(env.envdir, env.envname)
    print('using "%(envname)s" at "%(envdir)s" as virtualenv' % env)


def _virtualenv(command):
    with context_managers.cd(env.envpath):
        result = local('. bin/activate && ' + command)
    return result


def clone():
    _get_virtualenv_location()
    with context_managers.cd(env.envdir):
        local('virtualenv --no-site-packages --clear %(envname)s' % env)
    print _virtualenv('echo $PATH')
    _virtualenv('easy_install pip')
    print _virtualenv('pip install django==1.1.1')

It took me a while to figure out how I can issue commands within a virtual environment. Since fabric commands don’t share state sourcing activate has no effect on subsequent commands. This SO entry helped me to write _virtualenv() function. It is kind of ugly making all functions but fab commands private. I think if fab used __all__ or something similar it would be more explicit. Also a contrib module for virtualenv would be nice³.

Fabric has cool features such as failure handling and code/config editing. It is a great tool to create repeatable deployments. It feels great to be coding in python (well, to some extend). Perhaps the resulting code is a little too complex for local operations. I wish I could write a fab command that handles both local and remote deployments. I think it is not unusual to deploy on the same machine in another location. But having to SSH into localhost is weird, don’t you think?

I am sure there are better ways to accomplish the goal of the script above. Maybe there is an entirely different way to integrage fabric, virtualenv and pip. So, comments and suggestions are welcome as usual.

Maybe I’ll revisit buildout again as well.

1: You need to bootstrap (install) buildout with each development/deployment site.

2: About one third of it I think.

3: I would happily attempt one, once I learn fabric a little better.

Related posts:

1. How to Install MySQL with Fabric
2. Sad State of Web Development Industry in Türkiye
3. Dynamic Translation Apps for Django