marcuskammer/emacs.d
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
emacs.d/clones/lisp/stevelosh.com/blog/2011/06/django-advice/index.html
Marcus Kammer 7e48bb96c8
Update clones
2022-10-07 15:47:14 +02:00

982 lines
No EOL
75 KiB
HTML
Raw Blame History

<!DOCTYPE html>
<html lang='en'><head><meta charset='utf-8' /><meta name='pinterest' content='nopin' /><link href='../../../../static/css/style.css' rel='stylesheet' type='text/css' /><link href='../../../../static/css/print.css' rel='stylesheet' type='text/css' media='print' /><title>Django Advice / Steve Losh</title></head><body><header><a id='logo' href='https://stevelosh.com/'>Steve Losh</a><nav><a href='../../../index.html'>Blog</a> - <a href='https://stevelosh.com/projects/'>Projects</a> - <a href='https://stevelosh.com/photography/'>Photography</a> - <a href='https://stevelosh.com/links/'>Links</a> - <a href='https://stevelosh.com/rss.xml'>Feed</a></nav></header><hr class='main-separator' /><main id='page-blog-entry'><article><h1><a href='index.html'>Django Advice</a></h1><p class='date'>Posted on June 30th, 2011.</p><p>For the past year and a half or so I've been working full-time at <a href="http://dwaiter.com/">Dumbwaiter
Design</a> doing <a href="http://djangoproject.com/">Django</a> development. I've picked up a bunch of useful tricks along
the way that help me work, and I figured I'd share them.</p>
<p>I'm sure there are better ways to do some of the things that I mention. If you know
of any feel free to hit me up on <a href="http://twitter.com/stevelosh">Twitter</a> and let me know.</p>
<p>Also: this entry was written over several months, so if there are inconsistencies let
me know and I'll try to fix them.</p>
<ol class="table-of-contents"><li><a href="index.html#s1-vagrant">Vagrant</a><ol><li><a href="index.html#s2-why-vagrant">Why Vagrant?</a></li><li><a href="index.html#s3-using-fabric-to-stay-fast-and-automate-everything">Using Fabric to Stay Fast and Automate Everything</a></li></ol></li><li><a href="index.html#s4-wrangling-databases-with-south">Wrangling Databases with South</a><ol><li><a href="index.html#s5-useful-fabric-tasks">Useful Fabric Tasks</a></li></ol></li><li><a href="index.html#s6-watching-for-changes">Watching for Changes</a><ol><li><a href="index.html#s7-using-the-werkzeug-debugger-with-gunicorn">Using the Werkzeug Debugger with Gunicorn</a></li><li><a href="index.html#s8-pulling-uploads">Pulling Uploads</a></li><li><a href="index.html#s9-preventing-accidents">Preventing Accidents</a></li></ol></li><li><a href="index.html#s10-working-with-third-party-apps">Working with Third-Party Apps</a><ol><li><a href="index.html#s11-installing-apps-from-repositories">Installing Apps from Repositories</a></li><li><a href="index.html#s12-mirroring-repositories">Mirroring Repositories</a></li><li><a href="index.html#s13-using-bcvi-to-edit-files">Using BCVI to Edit Files</a></li></ol></li><li><a href="index.html#s14-improving-the-admin-interface">Improving the Admin Interface</a><ol><li><a href="index.html#s15-enter-grappelli">Enter Grappelli</a></li><li><a href="index.html#s16-an-ugly-hack-to-show-usable-foreign-key-fields">An Ugly Hack to Show Usable Foreign Key Fields</a></li></ol></li><li><a href="index.html#s17-using-django-annoying">Using Django-Annoying</a><ol><li><a href="index.html#s18-the-render-to-decorator">The render_to Decorator</a></li><li><a href="index.html#s19-the-ajax-request-decorator">The ajax_request Decorator</a></li></ol></li><li><a href="index.html#s20-templating-tricks">Templating Tricks</a><ol><li><a href="index.html#s21-null-checks-and-fallbacks">Null Checks and Fallbacks</a></li><li><a href="index.html#s22-manipulating-query-strings">Manipulating Query Strings</a></li><li><a href="index.html#s23-satisfying-your-designer-with-typogrify">Satisfying Your Designer with Typogrify</a></li></ol></li><li><a href="index.html#s24-the-flat-page-trainwreck">The Flat Page Trainwreck</a></li><li><a href="index.html#s25-editing-with-vim">Editing with Vim</a><ol><li><a href="index.html#s26-vim-for-django">Vim for Django</a></li><li><a href="index.html#s27-filetype-mappings">Filetype Mappings</a></li><li><a href="index.html#s28-python-sanity-checking">Python Sanity Checking</a></li><li><a href="index.html#s29-javascript-sanity-checking-and-folding">Javascript Sanity Checking and Folding</a></li><li><a href="index.html#s30-django-autocommands">Django Autocommands</a></li></ol></li><li><a href="index.html#s31-conclusion">Conclusion</a></li></ol>
<h2 id="s1-vagrant"><a href="index.html#s1-vagrant">Vagrant</a></h2>
<p>I used to develop Django sites by running them on my OS X laptop locally and
deploying to a Linode VPS. I had a whole section of this post written up about
tricks and tips for working with that setup.</p>
<p>Then I found <a href="http://vagrantup.com/">Vagrant</a>.</p>
<p>I just deleted the entire section of this post I wrote.</p>
<p>Vagrant gives you a better way of working. You need to use it.</p>
<h3 id="s2-why-vagrant"><a href="index.html#s2-why-vagrant">Why Vagrant?</a></h3>
<p>If you haven't used it before, Vagrant is basically a tool for managing
<a href="http://www.virtualbox.org/">VirtualBox</a> VMs. It makes it easy to start, pause, and resume VMs. Instead of
installing Django in a virtualenv and developing against that, you run a VM which
runs your site and develop against that.</p>
<p>This may not sound like much, but it's kind of a big deal. The critical difference
is that you can now develop against the same setup that you'll be using in
production.</p>
<p>This cuts out a huge amount of pain that stems from OS differences. Here are a few
examples off the top of my head:</p>
<ul>
<li>URLField and MacPorts Python 2.5 on OS X. There's a <a href="https://trac.macports.org/ticket/24421">bug</a> where using
verify_exists will crash your site every time you save a model, unless you set
a particular environment variable with no debug information. Yeah, I spent
a couple of hours tracking that one down at work. Awesome.</li>
<li>Installing PIL on OS X is no picnic. <a href="http://mxcl.github.com/homebrew/">homebrew</a> makes things better, if you use it,
so this one isn't a huge deal.</li>
<li>Every time you update Python in-place on your local machines, ALL of your
virtualenvs break because the Python binaries inside are linked against global
Python library files. Have fun recreating them. I hope you froze your
<code>requirements.txt</code> files before you updated.</li>
</ul>
<p>Using Vagrant and VMs means you can just worry about ONE operating system and its
quirks. It saves you a ton of time.</p>
<p>Aside from that, there's another benefit to using Vagrant: it strongly encourages you
to learn and use an automated provisioning system. Support for Puppet and Chef is
built in. I chose Puppet, but if you prefer Chef that's cool too.</p>
<p>You can also use other tools like Fabric or some simple scripts, but I'd strongly
recommend giving Puppet or Chef a fair shot. It's a lot to learn, but they're both
widely tested and very powerful.</p>
<p>Because you're developing against a VM and deploying to a VM, you can reuse 90% of
the provisioning code across the two.</p>
<p>When I make a new site, I do the following to initialize a new Vagrant VM:</p>
<ol>
<li><code>vagrant up</code> (which runs Puppet to initialize the VM)</li>
<li><code>fab dev bootstrap</code></li>
</ol>
<p>When I'm ready to go live, I do the following:</p>
<ol>
<li>Buy a Linode VPS.</li>
<li>Run Puppet to initialize the VPS.</li>
<li>Enter the Linode info in my fabfile.</li>
<li><code>fab prod bootstrap</code></li>
</ol>
<p>No more screwing around with different paths, different versions of Nginx, different
versions of Python. When I'm developing something I can be pretty confident it will
&quot;just work&quot; in production without any major surprises.</p>
<h3 id="s3-using-fabric-to-stay-fast-and-automate-everything"><a href="index.html#s3-using-fabric-to-stay-fast-and-automate-everything">Using Fabric to Stay Fast and Automate Everything</a></h3>
<p>One of the problems with this setup is that I can't just run <code>python manage.py
whatever</code> any more because I need it to run on the VM.</p>
<p>To get around this I've created many simple <a href="http://fabfile.org/">Fabric</a> tasks to automate the common
things I need to do. Fabric is an awesome little Python utility for scripting tasks
(like deployments). We use it constantly at Dumbwaiter. Here are a few examples
from our fabfiles.</p>
<p>This first set is for running abitrary commands easily.</p>
<p><code>cmd</code> and <code>vcmd</code> will <code>cd</code> into the site directory on the VM and run a command of my
choosing. <code>vcmd</code> will prefix the command with the path to the virtualenv's <code>bin</code>
directory, so I can do something like <code>fab dev vcmd</code>, <code>pip install markdown</code>.</p>
<p>The <code>sdo</code> commands do the same thing, but <code>sudo</code>'ed.</p>
<pre><code><span class="code"><span class="special">def</span><span class="keyword"> cmd</span><span class="paren1">(<span class="code">cmd=<span class="string">&quot;&quot;</span></span>)</span>:
<span class="string">'''Run a command in the site directory. Usable from other commands or the CLI.'''</span>
require<span class="paren1">(<span class="code"><span class="string">'site_path'</span></span>)</span>
<span class="symbol">if</span> <span class="symbol">not</span> cmd:
sys.stdout.write<span class="paren1">(<span class="code">_cyan<span class="paren2">(<span class="code"><span class="string">&quot;Command to run: &quot;</span></span>)</span></span>)</span>
cmd = raw_input<span class="paren1">(<span class="code"></span>)</span>.strip<span class="paren1">(<span class="code"></span>)</span>
<span class="symbol">if</span> cmd:
with cd<span class="paren1">(<span class="code">env.site_path</span>)</span>:
run<span class="paren1">(<span class="code">cmd</span>)</span>
<span class="special">def</span><span class="keyword"> sdo</span><span class="paren1">(<span class="code">cmd=<span class="string">&quot;&quot;</span></span>)</span>:
<span class="string">'''Sudo a command in the site directory. Usable from other commands or the CLI.'''</span>
require<span class="paren1">(<span class="code"><span class="string">'site_path'</span></span>)</span>
<span class="symbol">if</span> <span class="symbol">not</span> cmd:
sys.stdout.write<span class="paren1">(<span class="code">_cyan<span class="paren2">(<span class="code"><span class="string">&quot;Command to run: sudo &quot;</span></span>)</span></span>)</span>
cmd = raw_input<span class="paren1">(<span class="code"></span>)</span>.strip<span class="paren1">(<span class="code"></span>)</span>
<span class="symbol">if</span> cmd:
with cd<span class="paren1">(<span class="code">env.site_path</span>)</span>:
sudo<span class="paren1">(<span class="code">cmd</span>)</span>
<span class="special">def</span><span class="keyword"> vcmd</span><span class="paren1">(<span class="code">cmd=<span class="string">&quot;&quot;</span></span>)</span>:
<span class="string">'''Run a virtualenv-based command in the site directory. Usable from other commands or the CLI.'''</span>
require<span class="paren1">(<span class="code"><span class="string">'site_path'</span></span>)</span>
require<span class="paren1">(<span class="code"><span class="string">'venv_path'</span></span>)</span>
<span class="symbol">if</span> <span class="symbol">not</span> cmd:
sys.stdout.write<span class="paren1">(<span class="code">_cyan<span class="paren2">(<span class="code"><span class="string">&quot;Command to run: %s/bin/&quot;</span> % env.venv_path.rstrip<span class="paren3">(<span class="code"><span class="string">'/'</span></span>)</span></span>)</span></span>)</span>
cmd = raw_input<span class="paren1">(<span class="code"></span>)</span>.strip<span class="paren1">(<span class="code"></span>)</span>
<span class="symbol">if</span> cmd:
with cd<span class="paren1">(<span class="code">env.site_path</span>)</span>:
run<span class="paren1">(<span class="code">env.venv_path.rstrip<span class="paren2">(<span class="code"><span class="string">'/'</span></span>)</span> + <span class="string">'/bin/'</span> + cmd</span>)</span>
<span class="special">def</span><span class="keyword"> vsdo</span><span class="paren1">(<span class="code">cmd=<span class="string">&quot;&quot;</span></span>)</span>:
<span class="string">'''Sudo a virtualenv-based command in the site directory. Usable from other commands or the CLI.'''</span>
require<span class="paren1">(<span class="code"><span class="string">'site_path'</span></span>)</span>
require<span class="paren1">(<span class="code"><span class="string">'venv_path'</span></span>)</span>
<span class="symbol">if</span> <span class="symbol">not</span> cmd:
sys.stdout.write<span class="paren1">(<span class="code">_cyan<span class="paren2">(<span class="code"><span class="string">&quot;Command to run: sudo %s/bin/&quot;</span> % env.venv_path.rstrip<span class="paren3">(<span class="code"><span class="string">'/'</span></span>)</span></span>)</span></span>)</span>
cmd = raw_input<span class="paren1">(<span class="code"></span>)</span>.strip<span class="paren1">(<span class="code"></span>)</span>
<span class="symbol">if</span> cmd:
with cd<span class="paren1">(<span class="code">env.site_path</span>)</span>:
sudo<span class="paren1">(<span class="code">env.venv_path.rstrip<span class="paren2">(<span class="code"><span class="string">'/'</span></span>)</span> + <span class="string">'/bin/'</span> + cmd</span>)</span></span></code></pre>
<p>This next set is just some common commands that I need to run often.</p>
<pre><code><span class="code"><span class="special">def</span><span class="keyword"> syncdb</span><span class="paren1">(<span class="code"></span>)</span>:
<span class="string">'''Run syncdb.'''</span>
require<span class="paren1">(<span class="code"><span class="string">'site_path'</span></span>)</span>
require<span class="paren1">(<span class="code"><span class="string">'venv_path'</span></span>)</span>
with cd<span class="paren1">(<span class="code">env.site_path</span>)</span>:
run<span class="paren1">(<span class="code">_python<span class="paren2">(<span class="code"><span class="string">'manage.py syncdb --noinput'</span></span>)</span></span>)</span>
<span class="special">def</span><span class="keyword"> collectstatic</span><span class="paren1">(<span class="code"></span>)</span>:
<span class="string">'''Collect static media.'''</span>
require<span class="paren1">(<span class="code"><span class="string">'site_path'</span></span>)</span>
require<span class="paren1">(<span class="code"><span class="string">'venv_path'</span></span>)</span>
with cd<span class="paren1">(<span class="code">env.site_path</span>)</span>:
sudo<span class="paren1">(<span class="code">_python<span class="paren2">(<span class="code"><span class="string">'manage.py collectstatic --noinput'</span></span>)</span></span>)</span>
<span class="special">def</span><span class="keyword"> rebuild_index</span><span class="paren1">(<span class="code"></span>)</span>:
<span class="string">'''Rebuild the search index.'''</span>
require<span class="paren1">(<span class="code"><span class="string">'site_path'</span></span>)</span>
require<span class="paren1">(<span class="code"><span class="string">'venv_path'</span></span>)</span>
require<span class="paren1">(<span class="code"><span class="string">'process_owner'</span></span>)</span>
with cd<span class="paren1">(<span class="code">env.site_path</span>)</span>:
sudo<span class="paren1">(<span class="code">_python<span class="paren2">(<span class="code"><span class="string">'manage.py rebuild_index'</span></span>)</span></span>)</span>
sudo<span class="paren1">(<span class="code"><span class="string">'chown -R %s .xapian'</span> % env.process_owner</span>)</span>
<span class="special">def</span><span class="keyword"> update_index</span><span class="paren1">(<span class="code"></span>)</span>:
<span class="string">'''Update the search index.'''</span>
require<span class="paren1">(<span class="code"><span class="string">'site_path'</span></span>)</span>
require<span class="paren1">(<span class="code"><span class="string">'venv_path'</span></span>)</span>
require<span class="paren1">(<span class="code"><span class="string">'process_owner'</span></span>)</span>
with cd<span class="paren1">(<span class="code">env.site_path</span>)</span>:
sudo<span class="paren1">(<span class="code">_python<span class="paren2">(<span class="code"><span class="string">'manage.py update_index'</span></span>)</span></span>)</span>
sudo<span class="paren1">(<span class="code"><span class="string">'chown -R %s .xapian'</span> % env.process_owner</span>)</span></span></code></pre>
<p>We also use Fabric to automate some of the more complex things we need to do.</p>
<p>This task <code>curl</code>'s the site's home page to make sure we haven't completely borked
things. We use it in lots of other tasks as a sanity check.</p>
<pre><code><span class="code"><span class="special">def</span><span class="keyword"> check</span><span class="paren1">(<span class="code"></span>)</span>:
<span class="string">'''Check that the home page of the site returns an HTTP 200.'''</span>
require<span class="paren1">(<span class="code"><span class="string">'site_url'</span></span>)</span>
<span class="symbol">print</span><span class="paren1">(<span class="code"><span class="string">'Checking site status...'</span></span>)</span>
<span class="symbol">if</span> <span class="symbol">not</span> <span class="string">'200 OK'</span> <span class="symbol">in</span> local<span class="paren1">(<span class="code"><span class="string">'curl --silent -I &quot;%s&quot;'</span> % env.site_url, capture=True</span>)</span>:
_sad<span class="paren1">(<span class="code"></span>)</span>
else:
_happy<span class="paren1">(<span class="code"></span>)</span></span></code></pre>
<p>The <code>_happy</code> and <code>_sad</code> functions just print out some simple messages to get our
attention:</p>
<pre><code><span class="code"><span class="symbol">from</span> fabric.colors <span class="symbol">import</span> red, green
<span class="special">def</span><span class="keyword"> _happy</span><span class="paren1">(<span class="code"></span>)</span>:
<span class="symbol">print</span><span class="paren1">(<span class="code">green<span class="paren2">(<span class="code"><span class="string">'</span><span class="string">\n</span><span class="string">Looks good from here!</span><span class="string">\n</span><span class="string">'</span></span>)</span></span>)</span>
<span class="special">def</span><span class="keyword"> _sad</span><span class="paren1">(<span class="code"></span>)</span>:
<span class="symbol">print</span><span class="paren1">(<span class="code">red<span class="paren2">(<span class="code">r<span class="string">'''
___ ___
/ /</span><span class="string">\ </span><span class="string"> /__/</span><span class="string">\
</span><span class="string"> / /::</span><span class="string">\ </span><span class="string"> </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\
</span><span class="string"> / /:/</span><span class="string">\:</span><span class="string">\ </span><span class="string"> </span><span class="string">\_</span><span class="string">_</span><span class="string">\:</span><span class="string">\
</span><span class="string"> / /:/ </span><span class="string">\:</span><span class="string">\ </span><span class="string"> ___ / /::</span><span class="string">\
</span><span class="string"> /__/:/ </span><span class="string">\_</span><span class="string">_</span><span class="string">\:</span><span class="string">\ </span><span class="string">/__/</span><span class="string">\ </span><span class="string"> /:/</span><span class="string">\:</span><span class="string">\
</span><span class="string"> </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\ </span><span class="string">/ /:/ </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\/</span><span class="string">:/__</span><span class="string">\/</span><span class="string">
</span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\ </span><span class="string"> /:/ </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">:/
</span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\/</span><span class="string">:/ </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\
</span><span class="string"> </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">:/ </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\
</span><span class="string"> </span><span class="string">\_</span><span class="string">_</span><span class="string">\/</span><span class="string"> </span><span class="string">\_</span><span class="string">_</span><span class="string">\/</span><span class="string">
___ ___ ___ ___
/__/</span><span class="string">\ </span><span class="string"> / /</span><span class="string">\ </span><span class="string"> / /</span><span class="string">\ </span><span class="string"> / /</span><span class="string">\ </span><span class="string"> ___
</span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\ </span><span class="string"> / /::</span><span class="string">\ </span><span class="string"> / /:/_ / /:/_ /__/</span><span class="string">\
</span><span class="string"> </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\ </span><span class="string"> / /:/</span><span class="string">\:</span><span class="string">\ </span><span class="string"> / /:/ /</span><span class="string">\ </span><span class="string"> / /:/ /</span><span class="string">\ </span><span class="string"> </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\
</span><span class="string"> _____</span><span class="string">\_</span><span class="string">_</span><span class="string">\:</span><span class="string">\ </span><span class="string"> / /:/ </span><span class="string">\:</span><span class="string">\ </span><span class="string"> / /:/ /:/_ / /:/ /::</span><span class="string">\ </span><span class="string"> </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\
</span><span class="string"> /__/::::::::</span><span class="string">\ </span><span class="string">/__/:/ </span><span class="string">\_</span><span class="string">_</span><span class="string">\:</span><span class="string">\ </span><span class="string">/__/:/ /:/ /</span><span class="string">\ </span><span class="string">/__/:/ /:/</span><span class="string">\:</span><span class="string">\ </span><span class="string"> </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\
</span><span class="string"> </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\~</span><span class="string">~</span><span class="string">\~</span><span class="string">~</span><span class="string">\/</span><span class="string"> </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\ </span><span class="string">/ /:/ </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\/</span><span class="string">:/ /:/ </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\/</span><span class="string">:/~/:/ </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\
</span><span class="string"> </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\ </span><span class="string"> ~~~ </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\ </span><span class="string"> /:/ </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">:/ /:/ </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">:/ /:/ </span><span class="string">\_</span><span class="string">_</span><span class="string">\/</span><span class="string">
</span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\ </span><span class="string"> </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\/</span><span class="string">:/ </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\/</span><span class="string">:/ </span><span class="string">\_</span><span class="string">_</span><span class="string">\/</span><span class="string"> /:/ __
</span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">\ </span><span class="string"> </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">:/ </span><span class="string">\ </span><span class="string"> </span><span class="string">\:</span><span class="string">:/ /__/:/ /__/</span><span class="string">\
</span><span class="string"> </span><span class="string">\_</span><span class="string">_</span><span class="string">\/</span><span class="string"> </span><span class="string">\_</span><span class="string">_</span><span class="string">\/</span><span class="string"> </span><span class="string">\_</span><span class="string">_</span><span class="string">\/</span><span class="string"> </span><span class="string">\_</span><span class="string">_</span><span class="string">\/</span><span class="string"> </span><span class="string">\_</span><span class="string">_</span><span class="string">\/</span><span class="string">
Something seems to have gone wrong!
You should probably take a look at that.
'''</span></span>)</span></span>)</span></span></code></pre>
<p>This one is for when <code>python manage.py reset APP</code> is broken because you've changed
some <code>db_column</code> names and Django chokes because of some constraits and you just want
to <strong>reset the fucking app</strong>.</p>
<p>It's the &quot;NUKE IT FROM ORBIT!!&quot; option.</p>
<pre><code><span class="code"><span class="special">def</span><span class="keyword"> KILL_IT_WITH_FIRE</span><span class="paren1">(<span class="code">app</span>)</span>:
require<span class="paren1">(<span class="code"><span class="string">'site_path'</span></span>)</span>
require<span class="paren1">(<span class="code"><span class="string">'venv_path'</span></span>)</span>
with cd<span class="paren1">(<span class="code">env.site_path</span>)</span>:
<span class="comment"># Generate and download the reset SQL.
</span> sudo<span class="paren1">(<span class="code">_python<span class="paren2">(<span class="code"><span class="string">'manage.py sqlreset %s &gt; reset.orig.sql'</span> % app</span>)</span></span>)</span>
get<span class="paren1">(<span class="code"><span class="string">'reset.orig.sql'</span></span>)</span>
with open<span class="paren1">(<span class="code"><span class="string">'reset.sql'</span>, <span class="string">'w'</span></span>)</span> as f:
with open<span class="paren1">(<span class="code"><span class="string">'reset.orig.sql'</span></span>)</span> as orig:
<span class="comment"># Step through the first chunk of the file (the &quot;drop&quot; part).
</span> line = orig.readline<span class="paren1">(<span class="code"></span>)</span>
<span class="symbol">while</span> <span class="symbol">not</span> line.startswith<span class="paren1">(<span class="code"><span class="string">'CREATE'</span></span>)</span>:
<span class="symbol">if</span> <span class="string">'CONSTRAINT'</span> <span class="symbol">in</span> line:
<span class="comment"># Don't write out CONSTRAINT lines.
</span> <span class="comment"># They're a problem when you change db_colum names.
</span> <span class="symbol">pass</span>
<span class="symbol">elif</span> <span class="string">'DROP TABLE'</span> <span class="symbol">in</span> line:
<span class="comment"># Cascade drops.
</span> <span class="comment"># Hence with &quot;with fire&quot; part of this task's name.
</span> line = line<span class="paren1">[<span class="code">:-2</span>]</span> + <span class="string">' CASCADE;</span><span class="string">\n</span><span class="string">'</span>
f.write<span class="paren1">(<span class="code">line</span>)</span>
else:
<span class="comment"># Write other lines through untoched.
</span> f.write<span class="paren1">(<span class="code">line</span>)</span>
line = orig.readline<span class="paren1">(<span class="code"></span>)</span>
<span class="comment"># Write out the rest of the file untouched.
</span> f.write<span class="paren1">(<span class="code">line</span>)</span>
f.write<span class="paren1">(<span class="code">orig.read<span class="paren2">(<span class="code"></span>)</span></span>)</span>
<span class="comment"># Upload the processed SQL file.
</span> put<span class="paren1">(<span class="code"><span class="string">'reset.sql'</span>, os.path.join<span class="paren2">(<span class="code">env.site_path, <span class="string">'reset.ready.sql'</span></span>)</span>, use_sudo=True</span>)</span>
with cd<span class="paren1">(<span class="code">env.site_path</span>)</span>:
<span class="comment"># Use the SQL to reset the app, and fake a migration.
</span> run<span class="paren1">(<span class="code">_python<span class="paren2">(<span class="code"><span class="string">'manage.py dbshell &lt; reset.ready.sql'</span></span>)</span></span>)</span>
sudo<span class="paren1">(<span class="code">_python<span class="paren2">(<span class="code"><span class="string">'manage.py migrate --fake --delete-ghost-migrations '</span> + app</span>)</span></span>)</span></span></code></pre>
<p>This task uses Mercurial's local tags to add a <code>production</code> or <code>staging</code> tag in your local
repository, so you can easy see where the production/staging servers are at
compared to your local repo.</p>
<pre><code><span class="code"><span class="special">def</span><span class="keyword"> retag</span><span class="paren1">(<span class="code"></span>)</span>:
<span class="string">'''Check which revision the site is at and update the local tag.
Useful if someone else has deployed (which makes your production/staging local
tag incorrect.
'''</span>
require<span class="paren1">(<span class="code"><span class="string">'site_path'</span>, provided_by=<span class="paren2">[<span class="code"><span class="string">'prod'</span>, <span class="string">'stag'</span></span>]</span></span>)</span>
require<span class="paren1">(<span class="code"><span class="string">'env_name'</span>, provided_by=<span class="paren2">[<span class="code"><span class="string">'prod'</span>, <span class="string">'stag'</span></span>]</span></span>)</span>
with cd<span class="paren1">(<span class="code">env.site_path</span>)</span>:
current = run<span class="paren1">(<span class="code"><span class="string">'hg id --rev . --quiet'</span></span>)</span>.strip<span class="paren1">(<span class="code"><span class="string">' </span><span class="string">\n</span><span class="string">+'</span></span>)</span>
local<span class="paren1">(<span class="code"><span class="string">'hg tag --local --force %s --rev %s'</span> % <span class="paren2">(<span class="code">env.env_name, current</span>)</span></span>)</span></span></code></pre>
<p>This task tails the Gunicorn logs on the server so you can quickly find out what's
happening when things blow up.</p>
<pre><code><span class="code"><span class="special">def</span><span class="keyword"> tailgun</span><span class="paren1">(<span class="code">follow=<span class="string">''</span></span>)</span>:
<span class="string">&quot;&quot;</span><span class="string">&quot;Tail the Gunicorn log file.&quot;</span><span class="string">&quot;&quot;</span>
require<span class="paren1">(<span class="code"><span class="string">'site_path'</span></span>)</span>
with cd<span class="paren1">(<span class="code">env.site_path</span>)</span>:
<span class="symbol">if</span> follow:
run<span class="paren1">(<span class="code"><span class="string">'tail -f .gunicorn.log'</span></span>)</span>
else:
run<span class="paren1">(<span class="code"><span class="string">'tail .gunicorn.log'</span></span>)</span></span></code></pre>
<p>We've got a lot of other tasks but they're pretty specific to our setup.</p>
<h2 id="s4-wrangling-databases-with-south"><a href="index.html#s4-wrangling-databases-with-south">Wrangling Databases with South</a></h2>
<p>If you're not using <a href="http://south.aeracode.org/">South</a>, you need to start. Now.</p>
<p>No, really, I'll wait. Take 30 minutes, try the <a href="http://south.aeracode.org/docs/tutorial/index.html">tutorial</a>, wrap your head
around it and come back. It's far more important than this blog post.</p>
<h3 id="s5-useful-fabric-tasks"><a href="index.html#s5-useful-fabric-tasks">Useful Fabric Tasks</a></h3>
<p>South is awesome but its commands are very long-winded. Here's the set of fabric
tasks I use to save quite a bit of typing:</p>
<pre><code><span class="code"><span class="special">def</span><span class="keyword"> migrate</span><span class="paren1">(<span class="code">args=<span class="string">''</span></span>)</span>:
<span class="string">'''Run any needed migrations.'''</span>
require<span class="paren1">(<span class="code"><span class="string">'site_path'</span></span>)</span>
require<span class="paren1">(<span class="code"><span class="string">'venv_path'</span></span>)</span>
with cd<span class="paren1">(<span class="code">env.site_path</span>)</span>:
sudo<span class="paren1">(<span class="code">_python<span class="paren2">(<span class="code"><span class="string">'manage.py migrate '</span> + args</span>)</span></span>)</span>
<span class="special">def</span><span class="keyword"> migrate_fake</span><span class="paren1">(<span class="code">args=<span class="string">''</span></span>)</span>:
<span class="string">'''Run any needed migrations with --fake.'''</span>
require<span class="paren1">(<span class="code"><span class="string">'site_path'</span></span>)</span>
require<span class="paren1">(<span class="code"><span class="string">'venv_path'</span></span>)</span>
with cd<span class="paren1">(<span class="code">env.site_path</span>)</span>:
sudo<span class="paren1">(<span class="code">_python<span class="paren2">(<span class="code"><span class="string">'manage.py migrate --fake '</span> + args</span>)</span></span>)</span>
<span class="special">def</span><span class="keyword"> migrate_reset</span><span class="paren1">(<span class="code">args=<span class="string">''</span></span>)</span>:
<span class="string">'''Run any needed migrations with --fake. No, seriously.'''</span>
require<span class="paren1">(<span class="code"><span class="string">'site_path'</span></span>)</span>
require<span class="paren1">(<span class="code"><span class="string">'venv_path'</span></span>)</span>
with cd<span class="paren1">(<span class="code">env.site_path</span>)</span>:
sudo<span class="paren1">(<span class="code">_python<span class="paren2">(<span class="code"><span class="string">'manage.py migrate --fake --delete-ghost-migrations '</span> + args</span>)</span></span>)</span></span></code></pre>
<p>Remember that running a migration without specifying an app will migrate everything,
so a simple <code>fab dev migrate</code> will do the trick.</p>
<h2 id="s6-watching-for-changes"><a href="index.html#s6-watching-for-changes">Watching for Changes</a></h2>
<p>When developing locally you'll want to make a change to your code and have the server
reload that code automatically. The Django development server does this, and we can
hack it into our Vagrant/Gunicorn setup too.</p>
<p>First, add a <code>monitor.py</code> file at the root of your project (I believe I found this
code <a href="http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode">here</a>, but I may be wrong):</p>
<pre><code><span class="code"><span class="symbol">import</span> os
<span class="symbol">import</span> sys
<span class="symbol">import</span> time
<span class="symbol">import</span> signal
<span class="symbol">import</span> threading
<span class="symbol">import</span> atexit
<span class="symbol">import</span> Queue
_interval = 1.0
_times = <span class="paren1">{<span class="code"></span>}</span>
_files = <span class="paren1">[<span class="code"></span>]</span>
_running = False
_queue = Queue.Queue<span class="paren1">(<span class="code"></span>)</span>
_lock = threading.Lock<span class="paren1">(<span class="code"></span>)</span>
<span class="special">def</span><span class="keyword"> _restart</span><span class="paren1">(<span class="code">path</span>)</span>:
_queue.put<span class="paren1">(<span class="code">True</span>)</span>
prefix = <span class="string">'monitor (pid=%d):'</span> % os.getpid<span class="paren1">(<span class="code"></span>)</span>
<span class="symbol">print</span> &gt;&gt; sys.stderr, <span class="string">'%s Change detected to </span><span class="string">\'</span><span class="string">%s</span><span class="string">\'</span><span class="string">.'</span> % <span class="paren1">(<span class="code">prefix, path</span>)</span>
<span class="symbol">print</span> &gt;&gt; sys.stderr, <span class="string">'%s Triggering process restart.'</span> % prefix
os.kill<span class="paren1">(<span class="code">os.getpid<span class="paren2">(<span class="code"></span>)</span>, signal.SIGINT</span>)</span>
<span class="special">def</span><span class="keyword"> _modified</span><span class="paren1">(<span class="code">path</span>)</span>:
try:
<span class="comment"># If path doesn't denote a file and were previously
</span> <span class="comment"># tracking it, then it has been removed or the file type
</span> <span class="comment"># has changed so force a restart. If not previously
</span> <span class="comment"># tracking the file then we can ignore it as probably
</span> <span class="comment"># pseudo reference such as when file extracted from a
</span> <span class="comment"># collection of modules contained in a zip file.
</span>
<span class="symbol">if</span> <span class="symbol">not</span> os.path.isfile<span class="paren1">(<span class="code">path</span>)</span>:
<span class="symbol">return</span> path <span class="symbol">in</span> _times
<span class="comment"># Check for when file last modified.
</span>
mtime = os.stat<span class="paren1">(<span class="code">path</span>)</span>.st_mtime
<span class="symbol">if</span> path <span class="symbol">not</span> <span class="symbol">in</span> _times:
_times<span class="paren1">[<span class="code">path</span>]</span> = mtime
<span class="comment"># Force restart when modification time has changed, even
</span> <span class="comment"># if time now older, as that could indicate older file
</span> <span class="comment"># has been restored.
</span>
<span class="symbol">if</span> mtime != _times<span class="paren1">[<span class="code">path</span>]</span>:
<span class="symbol">return</span> True
except:
<span class="comment"># If any exception occured, likely that file has been
</span> <span class="comment"># been removed just before stat(), so force a restart.
</span>
<span class="symbol">return</span> True
<span class="symbol">return</span> False
<span class="special">def</span><span class="keyword"> _monitor</span><span class="paren1">(<span class="code"></span>)</span>:
<span class="symbol">while</span> 1:
<span class="comment"># Check modification times on all files in sys.modules.
</span>
<span class="symbol">for</span> module <span class="symbol">in</span> sys.modules.values<span class="paren1">(<span class="code"></span>)</span>:
<span class="symbol">if</span> <span class="symbol">not</span> hasattr<span class="paren1">(<span class="code">module, <span class="string">'__file__'</span></span>)</span>:
<span class="symbol">continue</span>
path = getattr<span class="paren1">(<span class="code">module, <span class="string">'__file__'</span></span>)</span>
<span class="symbol">if</span> <span class="symbol">not</span> path:
<span class="symbol">continue</span>
<span class="symbol">if</span> os.path.splitext<span class="paren1">(<span class="code">path</span>)</span><span class="paren1">[<span class="code">1</span>]</span> <span class="symbol">in</span> <span class="paren1">[<span class="code"><span class="string">'.pyc'</span>, <span class="string">'.pyo'</span>, <span class="string">'.pyd'</span></span>]</span>:
path = path<span class="paren1">[<span class="code">:-1</span>]</span>
<span class="symbol">if</span> _modified<span class="paren1">(<span class="code">path</span>)</span>:
<span class="symbol">return</span> _restart<span class="paren1">(<span class="code">path</span>)</span>
<span class="comment"># Check modification times on files which have
</span> <span class="comment"># specifically been registered for monitoring.
</span>
<span class="symbol">for</span> path <span class="symbol">in</span> _files:
<span class="symbol">if</span> _modified<span class="paren1">(<span class="code">path</span>)</span>:
<span class="symbol">return</span> _restart<span class="paren1">(<span class="code">path</span>)</span>
<span class="comment"># Go to sleep for specified interval.
</span>
try:
<span class="symbol">return</span> _queue.get<span class="paren1">(<span class="code">timeout=_interval</span>)</span>
except:
<span class="symbol">pass</span>
_thread = threading.Thread<span class="paren1">(<span class="code">target=_monitor</span>)</span>
_thread.setDaemon<span class="paren1">(<span class="code">True</span>)</span>
<span class="special">def</span><span class="keyword"> _exiting</span><span class="paren1">(<span class="code"></span>)</span>:
try:
_queue.put<span class="paren1">(<span class="code">True</span>)</span>
except:
<span class="symbol">pass</span>
_thread.join<span class="paren1">(<span class="code"></span>)</span>
atexit.register<span class="paren1">(<span class="code">_exiting</span>)</span>
<span class="special">def</span><span class="keyword"> track</span><span class="paren1">(<span class="code">path</span>)</span>:
<span class="symbol">if</span> <span class="symbol">not</span> path <span class="symbol">in</span> _files:
_files.append<span class="paren1">(<span class="code">path</span>)</span>
<span class="special">def</span><span class="keyword"> start</span><span class="paren1">(<span class="code">interval=1.0</span>)</span>:
<span class="symbol">global</span> _interval
<span class="symbol">if</span> interval &lt; _interval:
_interval = interval
<span class="symbol">global</span> _running
_lock.acquire<span class="paren1">(<span class="code"></span>)</span>
<span class="symbol">if</span> <span class="symbol">not</span> _running:
prefix = <span class="string">'monitor (pid=%d):'</span> % os.getpid<span class="paren1">(<span class="code"></span>)</span>
<span class="symbol">print</span> &gt;&gt; sys.stderr, <span class="string">'%s Starting change monitor.'</span> % prefix
_running = True
_thread.start<span class="paren1">(<span class="code"></span>)</span>
_lock.release<span class="paren1">(<span class="code"></span>)</span></span></code></pre>
<p>Next add a <code>post_fork</code> hook to your Gunicorn config file that uses the monitor to
watch for changes:</p>
<pre><code><span class="code"><span class="special">def</span><span class="keyword"> post_fork</span><span class="paren1">(<span class="code">server, worker</span>)</span>:
<span class="symbol">import</span> monitor
<span class="symbol">import</span> local_settings
<span class="symbol">if</span> local_settings.DEBUG:
server.log.info<span class="paren1">(<span class="code"><span class="string">&quot;Starting change monitor.&quot;</span></span>)</span>
monitor.start<span class="paren1">(<span class="code">interval=1.0</span>)</span></span></code></pre>
<p>Now the Gunicorn server will automatically restart whenever code is changed. Use
whatever method for determining debug status that you like. We use
<code>local_settings.py</code> files which all have <code>DEBUG</code> variables, so that works for us.</p>
<p>It will <em>not</em> restart when you add new code (e.g. when you install a new app), so
you'll need to handle that manually with <code>fab dev restart</code>, but that's not too bad!</p>
<h3 id="s7-using-the-werkzeug-debugger-with-gunicorn"><a href="index.html#s7-using-the-werkzeug-debugger-with-gunicorn">Using the Werkzeug Debugger with Gunicorn</a></h3>
<p>The final piece of the puzzle is being able to use the fantastic <a href="http://werkzeug.pocoo.org/docs/debug/">Werkzeug
Debugger</a> while running on the development VM with Gunicorn.</p>
<p>To do this, create a <code>debug_wsgi.py</code> file at the root of your project:</p>
<pre><code><span class="code"><span class="symbol">import</span> os
<span class="symbol">import</span> sys
<span class="symbol">import</span> site
parent = os.path.dirname
site_dir = parent<span class="paren1">(<span class="code">os.path.abspath<span class="paren2">(<span class="code">__file__</span>)</span></span>)</span>
project_dir = parent<span class="paren1">(<span class="code">parent<span class="paren2">(<span class="code">os.path.abspath<span class="paren3">(<span class="code">__file__</span>)</span></span>)</span></span>)</span>
sys.path.insert<span class="paren1">(<span class="code">0, project_dir</span>)</span>
sys.path.insert<span class="paren1">(<span class="code">0, site_dir</span>)</span>
site.addsitedir<span class="paren1">(<span class="code"><span class="string">'VIRTUALENV_SITE_PACKAGES'</span></span>)</span>
<span class="symbol">from</span> django.core.management <span class="symbol">import</span> setup_environ
<span class="symbol">import</span> settings
setup_environ<span class="paren1">(<span class="code">settings</span>)</span>
<span class="symbol">import</span> django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler<span class="paren1">(<span class="code"></span>)</span>
<span class="symbol">from</span> werkzeug.debug <span class="symbol">import</span> DebuggedApplication
application = DebuggedApplication<span class="paren1">(<span class="code">application, evalex=True</span>)</span>
<span class="special">def</span><span class="keyword"> null_technical_500_response</span><span class="paren1">(<span class="code">request, exc_type, exc_value, tb</span>)</span>:
<span class="symbol">raise</span> exc_type, exc_value, tb
<span class="symbol">from</span> django.views <span class="symbol">import</span> debug
debug.technical_500_response = null_technical_500_response</span></code></pre>
<p>Have Gunicorn use this file to run your development server with <code>gunicorn
debug_wsgi:application</code>.</p>
<p>Make sure to replace <code>'VIRTUALENV_SITE_PACKAGES'</code> with the <em>full</em> path to your
virtualenv's <code>site_packages</code> directory. You might want to make this a setting in
a machine-specific settings file.</p>
<h3 id="s8-pulling-uploads"><a href="index.html#s8-pulling-uploads">Pulling Uploads</a></h3>
<p>Once you give a client access to a site they'll probably be uploading images (through
Django's built-in file uploading features or with <a href="http://code.google.com/p/django-filebrowser/">django-filebrowser</a>).</p>
<p>When you're making changes locally it's often useful to have these uploaded files on
your local VM, otherwise you end up with a bunch of broken images.</p>
<p>Here's a simple Fabric task that will pull down all the uploads from the server:</p>
<pre><code><span class="code"><span class="special">def</span><span class="keyword"> pull_uploads</span><span class="paren1">(<span class="code"></span>)</span>:
<span class="string">'''Copy the uploads from the site to your local machine.'''</span>
require<span class="paren1">(<span class="code"><span class="string">'uploads_path'</span></span>)</span>
sudo<span class="paren1">(<span class="code"><span class="string">'chmod -R a+r &quot;%s&quot;'</span> % env.uploads_path</span>)</span>
rsync_command = r<span class="string">&quot;&quot;</span><span class="string">&quot;rsync -av -e 'ssh -p %s' %s@%s:%s %s&quot;</span><span class="string">&quot;&quot;</span> % <span class="paren1">(<span class="code">
env.port,
env.user, env.host,
env.uploads_path.rstrip<span class="paren2">(<span class="code"><span class="string">'/'</span></span>)</span> + <span class="string">'/'</span>,
<span class="string">'media/uploads'</span>
</span>)</span>
<span class="symbol">print</span> local<span class="paren1">(<span class="code">rsync_command, capture=False</span>)</span></span></code></pre>
<p>You might be wondering about the line that strips <code>/</code> characters and then adds them
back in. <code>rsync</code> does different things depending on whether you end a path with
a <code>/</code>, so this is actually pretty important.</p>
<p>In your host task you'll need to set the <code>uploads_path</code> variable to something like
this:</p>
<pre><code><span class="code"><span class="symbol">import</span> os
env.site_path = os.path.join<span class="paren1">(<span class="code"><span class="string">'var'</span>, <span class="string">'www'</span>, <span class="string">'myproject'</span></span>)</span>
env.uploads_path = os.path.join<span class="paren1">(<span class="code">env.site_path, <span class="string">'media'</span>, <span class="string">'uploads'</span></span>)</span></span></code></pre>
<p>Now you can run <code>fab production pull_uploads</code> to pull down all the files people have
uploaded to the production server.</p>
<h3 id="s9-preventing-accidents"><a href="index.html#s9-preventing-accidents">Preventing Accidents</a></h3>
<p>Deploying to test and staging servers should be quick and easy. Deploying to
production servers should be harder to prevent people from accidentally doing it.</p>
<p>I've created a little function that I call before deploying to production servers.
It forces me to type in random words from the system word list before proceeding to
make sure I <em>really</em> know what I'm doing:</p>
<pre><code><span class="code"><span class="symbol">import</span> os, random
<span class="symbol">from</span> fabric.api <span class="symbol">import</span> *
<span class="symbol">from</span> fabric.operations <span class="symbol">import</span> prompt
<span class="symbol">from</span> fabric.utils <span class="symbol">import</span> abort
WORDLIST_PATHS = <span class="paren1">[<span class="code">os.path.join<span class="paren2">(<span class="code"><span class="string">'/'</span>, <span class="string">'usr'</span>, <span class="string">'share'</span>, <span class="string">'dict'</span>, <span class="string">'words'</span></span>)</span></span>]</span>
DEFAULT_MESSAGE = <span class="string">&quot;Are you sure you want to do this?&quot;</span>
WORD_PROMPT = <span class="string">' [%d/%d] Type &quot;%s&quot; to continue (^C quits): '</span>
<span class="special">def</span><span class="keyword"> prevent_horrible_accidents</span><span class="paren1">(<span class="code">msg=DEFAULT_MESSAGE, horror_rating=1</span>)</span>:
<span class="string">&quot;&quot;</span><span class="string">&quot;Prompt the user to enter random words to prevent doing something stupid.&quot;</span><span class="string">&quot;&quot;</span>
valid_wordlist_paths = <span class="paren1">[<span class="code">wp <span class="symbol">for</span> wp <span class="symbol">in</span> WORDLIST_PATHS <span class="symbol">if</span> os.path.exists<span class="paren2">(<span class="code">wp</span>)</span></span>]</span>
<span class="symbol">if</span> <span class="symbol">not</span> valid_wordlist_paths:
abort<span class="paren1">(<span class="code"><span class="string">'No wordlists found!'</span></span>)</span>
with open<span class="paren1">(<span class="code">valid_wordlist_paths<span class="paren2">[<span class="code">0</span>]</span></span>)</span> as wordlist_file:
words = wordlist_file.readlines<span class="paren1">(<span class="code"></span>)</span>
<span class="symbol">print</span> msg
<span class="symbol">for</span> i <span class="symbol">in</span> range<span class="paren1">(<span class="code">int<span class="paren2">(<span class="code">horror_rating</span>)</span></span>)</span>:
word = words<span class="paren1">[<span class="code">random.randint<span class="paren2">(<span class="code">0, len<span class="paren3">(<span class="code">words</span>)</span></span>)</span></span>]</span>.strip<span class="paren1">(<span class="code"></span>)</span>
p_msg = WORD_PROMPT % <span class="paren1">(<span class="code">i+1, horror_rating, word</span>)</span>
answer = prompt<span class="paren1">(<span class="code">p_msg, validate=r<span class="string">'^%s$'</span> % word</span>)</span></span></code></pre>
<p>You may need to adjust <code>WORDLIST_PATHS</code> if you're not on OS X.</p>
<h2 id="s10-working-with-third-party-apps"><a href="index.html#s10-working-with-third-party-apps">Working with Third-Party Apps</a></h2>
<p>One of the best parts about working with Django is that many problems have already
been solved and the solutions have been released as open-source applications.</p>
<p>We use quite a few open-source apps, and there are a couple of tricks I've learned to
make working with them easier.</p>
<h3 id="s11-installing-apps-from-repositories"><a href="index.html#s11-installing-apps-from-repositories">Installing Apps from Repositories</a></h3>
<p>If I'm going to use an open-source Django app in a project I'll almost always install
it as an editable repository on the VM with <code>pip install -e</code>.</p>
<p>Others may disagree with me on this, but I think it's the best way to work.</p>
<p>Often I'll find a bug that I think may be in one of the third-party apps I'm using.
Installing the apps as repositories makes it easy to read their source and figure out
if the bug is really in the app.</p>
<p>If the bug <em>is</em> in the third-party app having the app installed as a repository makes
it simple to fix the bug, fork the project on BitBucket or GitHub, send a pull
request, and get back to work.</p>
<h3 id="s12-mirroring-repositories"><a href="index.html#s12-mirroring-repositories">Mirroring Repositories</a></h3>
<p>One problem we've run into at Dumbwaiter is that the repos for third-party apps we
use are scattered across GitHub, BitBucket, Google Code, and other servers. If any
one of these services goes down we're stuck waiting for it to come back up.</p>
<p>A while ago I took half a day and consolidated all of these repos onto one of the
servers that we control. The basic process went like this:</p>
<ul>
<li>Use <a href="http://hg-git.github.com/">hg-git</a> and <a href="https://bitbucket.org/durin42/hgsubversion/wiki/Home">hgsubversion</a> to convert the git and SVN repos to Mercurial
repos.</li>
<li>Set up a master <code>mirror</code> Mercurial repo with all the app repos as subrepos.</li>
<li>Push the master repo and all the subrepos up to one of our Linodes.</li>
</ul>
<p>Now we can use <code>-e ssh://hg@OUR_LINODE/mirror/APP@REV_THAT_WORKS#egg=APP</code> in our
<code>requirements.txt</code> files to install apps from our mirror. When we want to update our
dependencies we can simply pull from the upstream repos and commit in the mirror
repo.</p>
<p>If our mirror goes down it's not a big deal, because we have far bigger problems to
worry about than new projects.</p>
<p>I wrote a few scripts to automate updating apps and such, but they're extremely hacky
so I don't want to post them here. Take half a day and write your own set — it's
definitely worth it to have your own mirror of your specific dependencies.</p>
<h3 id="s13-using-bcvi-to-edit-files"><a href="index.html#s13-using-bcvi-to-edit-files">Using BCVI to Edit Files</a></h3>
<p>I said that when I find a bug that I think is in a third-party app I'll poke around
with the app and try to figure it out. But since all the apps are installed in
a virtualenv on the Vagrant VM it might seem like it's a pain in the ass to edit
those files!</p>
<p>Luckily <a href="http://sshmenu.sourceforge.net/articles/bcvi/">BCVI</a> exists. It's a utility that opens a &quot;back channel&quot; to your local
machine when you SSH and lets you run <code>vi FILE</code> to open that file in
Vim/MacVim/GVim/etc on your <em>local</em> machine. When you save the file it uploads it
back to the server automatically for you.</p>
<p>It can be a bit tricky to set up, but it's worth it. Trust me.</p>
<h2 id="s14-improving-the-admin-interface"><a href="index.html#s14-improving-the-admin-interface">Improving the Admin Interface</a></h2>
<p>I'm going to be honest: Django's admin interface is the main reason I'm still using
it. Other frameworks like <a href="http://flask.pocoo.org/">Flask</a> are great, but Django's admin saves me
<em>ridiculous</em> amounts of time when I'm making simple CRUD sites for clients.</p>
<p>That said, the Django admin isn't the prettiest thing around, but we can give it
a facelift.</p>
<h3 id="s15-enter-grappelli"><a href="index.html#s15-enter-grappelli">Enter Grappelli</a></h3>
<p><a href="http://django-grappelli.readthedocs.org/">Grappelli</a> is a Django app that reskins the admin interface beautifully. It also
adds some functionality like drag-and-drop reordering of inlines, and allows you to
customize the dashboard to your liking. <em>Every</em> Django site I work on uses Grappelli
-- it's just that good.</p>
<p>The downside of Grappelli is that it changes quite a lot and breaks backwards
compatibility at the drop of a hat.</p>
<p>If you're going to use Grappelli you <em>must</em> freeze your requirements.txt files and
work with a single version at a time. Trying to always work from the trunk will make
you drink.</p>
<h3 id="s16-an-ugly-hack-to-show-usable-foreign-key-fields"><a href="index.html#s16-an-ugly-hack-to-show-usable-foreign-key-fields">An Ugly Hack to Show Usable Foreign Key Fields</a></h3>
<p>A limitation of both Grappelli and the stock Django admin is that it seems like you
can't easily show fields from related models in the admin list view.</p>
<p>For example, if you're new to Django you might expect this to work:</p>
<pre><code><span class="code"><span class="special">class</span><span class="keyword"> BlogEntryAdmin</span><span class="paren1">(<span class="code">admin.ModelAdmin</span>)</span>:
list_display = <span class="paren1">(<span class="code"><span class="string">'title'</span>, <span class="string">'author__name'</span></span>)</span></span></code></pre>
<p>Unfortunately Django chokes on the <code>author__name</code> lookup. You can <em>display</em> the name
without too much fuss:</p>
<pre><code><span class="code"><span class="special">class</span><span class="keyword"> BlogEntryAdmin</span><span class="paren1">(<span class="code">admin.ModelAdmin</span>)</span>:
list_display = <span class="paren1">(<span class="code"><span class="string">'title'</span>, <span class="string">'author_name'</span></span>)</span>
<span class="special">def</span><span class="keyword"> author_name</span><span class="paren1">(<span class="code">self, obj</span>)</span>:
<span class="symbol">return</span> obj.name</span></code></pre>
<p>That will display the name just fine. However, it won't be a fully-fledged column in
the Django admin because you can't sort on it.</p>
<p>It may seem like this is the end — if it could be a fully-functional field, why
wouldn't Django just let you use <code>author__name</code>? Luckily we can add one more line to
fix the problem:</p>
<pre><code><span class="code"><span class="special">class</span><span class="keyword"> BlogEntryAdmin</span><span class="paren1">(<span class="code">admin.ModelAdmin</span>)</span>:
list_display = <span class="paren1">(<span class="code"><span class="string">'title'</span>, <span class="string">'author_name'</span></span>)</span>
<span class="special">def</span><span class="keyword"> author_name</span><span class="paren1">(<span class="code">self, obj</span>)</span>:
<span class="symbol">return</span> obj.name
author_name.admin_order_field = <span class="string">'author__name'</span></span></code></pre>
<p>Now the author name has all the functionality of a real <code>list_display</code> entry.</p>
<h2 id="s17-using-django-annoying"><a href="index.html#s17-using-django-annoying">Using Django-Annoying</a></h2>
<p>If you haven't heard of <a href="https://bitbucket.org/offline/django-annoying/wiki/Home">django-annoying</a> you should definitely check it out. It's
got a bunch of miscellaneous functions that fix some common, annoying parts of
Django.</p>
<p>My two personal favorites from the package are a pair of decorators that help make
your views much, much cleaner.</p>
<h3 id="s18-the-render-to-decorator"><a href="index.html#s18-the-render-to-decorator">The render_to Decorator</a></h3>
<p>The decorator is called <code>render_to</code> and it eliminates the ugly <code>render_to_response</code>
calls that Django normally forces you to use in every single view.</p>
<p>Normally you'd use something like this:</p>
<pre><code><span class="code"><span class="special">def</span><span class="keyword"> videos</span><span class="paren1">(<span class="code">request</span>)</span>:
videos = Video.objects.all<span class="paren1">(<span class="code"></span>)</span>
<span class="symbol">return</span> render_to_response<span class="paren1">(<span class="code"><span class="string">'video_list.html'</span>, <span class="paren2">{<span class="code"> <span class="string">'videos'</span>: videos </span>}</span>,
context_instance=RequestContext<span class="paren2">(<span class="code">request</span>)</span></span>)</span></span></code></pre>
<p>With <code>render_to</code> your view gets much cleaner:</p>
<pre><code><span class="code"><span class="symbol">@render_to</span><span class="paren1">(<span class="code"><span class="string">'video_list.html'</span></span>)</span>
<span class="special">def</span><span class="keyword"> videos</span><span class="paren1">(<span class="code">request</span>)</span>:
videos = Video.objects.all<span class="paren1">(<span class="code"></span>)</span>
<span class="symbol">return</span> <span class="paren1">{<span class="code"> <span class="string">'videos'</span>: videos </span>}</span></span></code></pre>
<p>Less typing <code>context_instance=...</code> over and over, and less syntax to remember.</p>
<p>Yes, I know about Django 1.3's <code>render</code> shortcut. You have to type <code>request</code> every
single time with <code>render</code>, so the <code>render_to</code> decorator still wins.</p>
<h3 id="s19-the-ajax-request-decorator"><a href="index.html#s19-the-ajax-request-decorator">The ajax_request Decorator</a></h3>
<p>The <code>ajax_request</code> decorator is like <code>render_to</code> for AJAX requests. You simply
return a Python dictionary from your view and the decorator handles the JSON encoding
and such:</p>
<pre><code><span class="code"><span class="symbol">@ajax_request</span>
<span class="special">def</span><span class="keyword"> ajax_get_entries</span><span class="paren1">(<span class="code">request</span>)</span>:
blog_entries = BlogEntry.objects.all<span class="paren1">(<span class="code"></span>)</span>
<span class="symbol">return</span> <span class="paren1">{<span class="code"> <span class="string">'entries'</span>: <span class="paren2">[<span class="code"><span class="paren3">(<span class="code">entry.title, entry.get_absolute_url<span class="paren4">(<span class="code"></span>)</span></span>)</span>
<span class="symbol">for</span> entry <span class="symbol">in</span> entries</span>]</span></span>}</span></span></code></pre>
<h2 id="s20-templating-tricks"><a href="index.html#s20-templating-tricks">Templating Tricks</a></h2>
<p>I'm not a frontend developer, but I've done my share of HTML hacking at Dumbwaiter.
Here are a few of the tricks I've learned.</p>
<h3 id="s21-null-checks-and-fallbacks"><a href="index.html#s21-null-checks-and-fallbacks">Null Checks and Fallbacks</a></h3>
<p>A common pattern I see in Django templates looks like this:</p>
<pre><code>{% templatetag openblock %} if business.title {% templatetag closeblock %}
{% templatetag openvariable %} business.title {% templatetag closevariable %}
{% templatetag openblock %} else {% templatetag closeblock %}
{% templatetag openvariable %} business.short_title {% templatetag closevariable %}
{% templatetag openblock %} endif {% templatetag closeblock %}</code></pre>
<p>Here's a simpler way to do that:</p>
<pre><code>{% templatetag openblock %} firstof business.title business.short_title {% templatetag closeblock %}</code></pre>
<p><code>firstof</code> will return the first non-Falsy item in its arguments.</p>
<h3 id="s22-manipulating-query-strings"><a href="index.html#s22-manipulating-query-strings">Manipulating Query Strings</a></h3>
<p>Query strings are normally not a big deal, but every once in a while you'll have
a model listing page where you need to filter by category, and number of spaces, and
tags, etc all at once.</p>
<p>If you're trying to manage GET queries manually it can get pretty hairy very fast.</p>
<p><a href="http://djangosnippets.org/snippets/2237/">This Django snippet</a> makes working with query strings in templates
a breeze.</p>
<h3 id="s23-satisfying-your-designer-with-typogrify"><a href="index.html#s23-satisfying-your-designer-with-typogrify">Satisfying Your Designer with Typogrify</a></h3>
<p>If you haven't heard of <a href="http://code.google.com/p/typogrify/">Typogrify</a> you should take a look at it. It makes it easy
to add all the typographic goodness your designers are looking for.</p>
<h2 id="s24-the-flat-page-trainwreck"><a href="index.html#s24-the-flat-page-trainwreck">The Flat Page Trainwreck</a></h2>
<p>Creating a site for a client is very different than creating a site for yourself.
For pretty much every client we've dealt with we've heard: &quot;can't we just create
a new page at /drink-special/ for this special deal we're running?&quot;</p>
<p>Having clients go through you to make new pages is simply too much overhead. We
needed a way to let clients create new pages (like <code>/drink-special/</code>) on the fly,
without our intervention.</p>
<p>Django has a &quot;flatpages&quot; app that solves this problem. Kind of.</p>
<p>When using flat pages clients need to do two things that are often too much for
non-technical people:</p>
<ul>
<li>Manage URLs manually.</li>
<li>Write all content as raw HTML in a single text field.</li>
</ul>
<p>We've tried a lot of Django CMS apps at Dumbwaiter, and none of them made us happy.
They all seemed to have some or all of the following problems:</p>
<ul>
<li>They take over your site and make you write a &quot;Django-WhateverCMS site&quot; instead of
a &quot;Django site&quot;.</li>
<li>They're extremely feature-rich and complicated with features like
internationalization, redirects, versions, and many others. This is great if you
need the flexibility, but bad if your clients just need to create a couple of
pages.</li>
<li>They break <code>APPEND_TRAILING_SLASH</code> and make you clutter your <code>urls.py</code> files with
a bunch of extra code ot handle this.</li>
</ul>
<p>I finally got fed up and wrote my own Django CMS app: <a href="http://stoat.rtfd.org/">Stoat</a>. Stoat is designed
to be sleek, with only the features that our clients need.</p>
<p>It's not officially version 1.0 yet, but we're using it for a few clients and it's
working well. Check it out if you're looking for a more lightweight CMS app.</p>
<h2 id="s25-editing-with-vim"><a href="index.html#s25-editing-with-vim">Editing with Vim</a></h2>
<p>I <a href="https://stevelosh.com/blog/2010/09/coming-home-to-vim/">use Vim</a> to edit everything. Naturally I've found a bunch of plugins,
mappings and other tricks that make it even better when working on Django projects.</p>
<h3 id="s26-vim-for-django"><a href="index.html#s26-vim-for-django">Vim for Django</a></h3>
<p>There are a lot of ways to make Vim work with Django. I won't go into all of them in
this post, but a good place to start is <a href="https://code.djangoproject.com/wiki/UsingVimWithDjango">this Django wiki page</a>.</p>
<h3 id="s27-filetype-mappings"><a href="index.html#s27-filetype-mappings">Filetype Mappings</a></h3>
<p>Most files in a Django project have one of two extensions: <code>.py</code> and <code>.html</code>.
Unfortunately these extensions aren't unique to Django, so Vim doesn't automatically
set the correct <code>filetype</code> when you open one.</p>
<p>I've added a few mappings to my <code>.vimrc</code> to make it quick and easy to set the correct
<code>filetype</code>:</p>
<pre><code>nnoremap _dt :set ft=htmldjango&lt;CR&gt;
nnoremap _pd :set ft=python.django&lt;CR&gt;</code></pre>
<p>I also have a few autocommands that set the filetype for me when I'm editing a file
whose name &quot;sounds like&quot; a Django file:</p>
<pre><code>au BufNewFile,BufRead admin.py setlocal filetype=python.django
au BufNewFile,BufRead urls.py setlocal filetype=python.django
au BufNewFile,BufRead models.py setlocal filetype=python.django
au BufNewFile,BufRead views.py setlocal filetype=python.django
au BufNewFile,BufRead settings.py setlocal filetype=python.django
au BufNewFile,BufRead forms.py setlocal filetype=python.django</code></pre>
<h3 id="s28-python-sanity-checking"><a href="index.html#s28-python-sanity-checking">Python Sanity Checking</a></h3>
<p>Lets be honest here: it takes a lot of work to turn Vim into an &quot;IDE&quot;, and even then
it doesn't reach the level of something like Eclipse for Java. Anyone who claims it
has the same levels of integration and functionality is simply lying.</p>
<p>With that said I'll make an opinionated statement that is going to piss some of you
off.</p>
<p><strong>I am a programmer, not an IDE operator.</strong></p>
<p>I know Python.</p>
<p>I know Django.</p>
<p>I don't need to hit Cmd+Space twice for every line of code I write.</p>
<p>When someone asks me &quot;how do you run your site&quot; I do <strong>not</strong> answer: &quot;click the green
triangle in Eclipse&quot;.</p>
<p>However, I am human. I do stupid things like forgetting a colon or forgetting an
import. To help me with those problems I've turned to <a href="http://www.vim.org/scripts/script.php?script_id=2736">Syntastic</a> and Kevin
Watters' <a href="https://github.com/kevinw/pyflakes">Pyflakes fork</a> for Vim.</p>
<p>Syntastic is a Vim plugin that adds on-the-fly syntax-checking for many different
file formats. If you have Pyflakes installed it will automatically show you errors
in your code.</p>
<p>Pyflakes doesn't have IDE-level integration with your code. It doesn't check that
whatever libraries you <code>import</code> actually exist. It simply checks that your files are
probably-valid Python, and tells you when they're not.</p>
<p>This is enough for me. It catches the stupid mistakes I make. The less-stupid,
more-subtle mistakes slip by it, but to be fair many of them would have slipped by an
&quot;IDE&quot; as well.</p>
<h3 id="s29-javascript-sanity-checking-and-folding"><a href="index.html#s29-javascript-sanity-checking-and-folding">Javascript Sanity Checking and Folding</a></h3>
<p>Syntastic also supports Javascript if you have Javascript Lint installed (<code>brew
install jsl</code> on OS X). It's not perfect but it <em>will</em> catch things like using
trailing commas in object literals.</p>
<p>Some people like using CTags to get an overview of their code. I take a more
low-tech approach and am in love with code folding. When I fold my code
I automatically get an overview of everything in each file.</p>
<p>By default Vim doesn't fold Javascript files, but you can add some basic, perfectly
serviceable folding with these two lines in your .vimrc:</p>
<pre><code>au FileType javascript setlocal foldmethod=marker
au FileType javascript setlocal foldmarker={,}
</code></pre>
<h3 id="s30-django-autocommands"><a href="index.html#s30-django-autocommands">Django Autocommands</a></h3>
<p>I <em>rarely</em> work with raw HTML files any more. Whenever I open a file ending in
<code>.html</code> it's almost always a Django template (or a <a href="http://jinja.pocoo.org/">Jinja</a> template, which has
a very similar syntax). I've added an autocommand to automatically set the correct
filetype whenever I open a <code>.html</code> file:</p>
<pre><code>au BufNewFile,BufRead *.html setlocal filetype=htmldjango</code></pre>
<p>I also have some autocommands that tweak how a few specific files are handled:</p>
<pre><code>au BufNewFile,BufRead urls.py setlocal nowrap
au BufNewFile,BufRead settings.py normal! zR
au BufNewFile,BufRead dashboard.py normal! zR</code></pre>
<p>This automatically unfolds <code>urls.py</code>, <code>dashboard.py</code> and <code>settings.py</code> (I prefer
seeing those unfolded) and unsets line wrapping for <code>urls.py</code> (lines in a <code>urls.py</code>
file can get long and are hard to read when wrapped).</p>
<h2 id="s31-conclusion"><a href="index.html#s31-conclusion">Conclusion</a></h2>
<p>I hope that this longer-than-expected blog entry has given you at least one or two
things to think about.</p>
<p>I've learned a lot while working with Django for Dumbwaiter every day, but I'm sure
there's still a lot I've missed. If you see something I could be doing better please
let me know!</p>
</article></main><hr class='main-separator' /><footer><nav><a href='https://github.com/sjl/'>GitHub</a> ・ <a href='https://twitter.com/stevelosh/'>Twitter</a> ・ <a href='https://instagram.com/thirtytwobirds/'>Instagram</a> ・ <a href='https://hg.stevelosh.com/.plan/'>.plan</a></nav></footer></body></html>
Reference in a new issue View git blame Copy permalink