Virtualenvwrapper for your production server
Virtualenvwrapper is a popular tool for the Django developer who works on several different projects at the same time. I have not seen much on the web about how this tool can also help simplify your production setup, in particular your Fabric and crontab scripts. So here's a quick writeup...
From the author:
(Virtualenvwrapper)... includes wrappers for creating and deleting virtual environments and otherwise managing your development workflow, making it easier to work on more than one project at a time without introducing conflicts in their dependencies.
In fact, once you've set it up, switching from one Django project to another is as easy as:
I will admit that when a colleague first introduced me to this package, I couldn't really see the point; but then at the time I was working on a long contract, dealing with just a single site. In the year since finishing that job, I have worked on maybe 20 Django projects - often several in the same day - and I am now a complete convert to this wonderful tool!
I will not write anything more on setting up & using virtualenvwrapper; there are several excellent tutorials on this subject. Instead, I want to talk about...
Virtualenvwrapper for your production server
So why would you use virtualenvwrapper in production? The main justification - handling multiple Django projects in parallel - often does not apply, as you might have only one site (and hence one virtualenv) on a client's server. Well, I have 3 very good reasons:
1. Easy command-line management
A quick explanation about how our project is set up in Webfaction (and why our folder path might appear complicated):
- We have declared a Django webapp called hbcc_prod.
- Within it is a virtualenv called hartsbourneCC.
- Within that is our project, also called hartsbourneCC.
Instead of remembering that long path, with virtualenvwrapper we can simply type:
... and not have to remember paths. Ok, that's not a killer feature... let's try something else!
If you have set up a virtualenv, you might use something like:
from fabric.context_managers import prefix, cd from fabric.api import env, run def collect_static_files(*args): with cd('/home/adoro/webapps/hbcc_prod/hartsbourneCC/hartsbourneCC'): with prefix('source ../bin/activate')): run('./manage.py collectstatic --noinput -v0')
So our function is:
- Changing directories to the project directory.
- Activating the virtualenv.
- Running the command.
Now, if we are using virtualenvwrapper, we could write:
from fabric.context_managers import prefix from fabric.api import env, run def collect_static_files(*args): with prefix('workon hartsbourneCC'): run('./manage.py collectstatic --noinput -v0')
Which is shorter & easier to read. Also, we are now following good DRY (Don't Repeat Yourself) principles, by having the location of the project, and of its virtualenv, defined in only one place - in the definition of the virtualenvwrapper on the server. We no longer store (duplicate) this information in our Fabric script.
Note: the above example hardcodes the name of the virtualenv in the function; a more (re)usable piece of code would store this information in the
env environment dictionary.
If we have a regular job to run for our project, we might write the following crontab command:
0 6 * * * cd /home/adoro/webapps/hbcc_prod/hartsbourneCC/hartsbourneCC && source ../bin/activate && ./manage.py cleanup
If we are using virtualenvwrapper, we can instead write:
SHELL=/bin/bash 0 0 * * * source $HOME/.bashrc && workon hartsbourneCC && ./manage.py cleanup
Once again, we have something that is more DRY-ish, as well as being (IMO) easier to read. A couple of explanations are required:
- SHELL is required to switch to a shell that will work with a virtualenvwrapper.
- source $HOME/.bashrc runs the virtualenvwrapper initialisation script (to be totally accurate, I should put this into its own script, and call that from either .bashrc when initialising a session, or when running a crontab line. But I'm lazy...).
But it don't work...
There is one downside to the above: at first, I would get error messages like:
Traceback (most recent call last): File "/usr/lib/python2.7/logging/handlers.py", line 78, in emit self.doRollover() File "/usr/lib/python2.7/logging/handlers.py", line 140, in doRollover os.remove(dfn) OSError: [Errno 2] No such file or directory: '/home/adoro/.virtualenvs/hook.log.1' Logged from file hook_loader.py, line 134
It turns out that virtualenwrapper does not like at all having 2 environments initialised at the exact same moment, which was happening when you have 2 crontabs running at the same minute. So if this is likely to happen, add a random
sleep in front of one of the jobs, as a quick-n-dirty way of avoiding clashes:
SHELL=/bin/bash */10 * * * * sleep 5; source $HOME/.bashrc && workon hartsbourneCC && ./manage.py run_polling 0 0 * * * source $HOME/.bashrc && workon hartsbourneCC && ./manage.py cleanup