291 lines
19 KiB
HTML
291 lines
19 KiB
HTML
|
<!DOCTYPE html>
|
||
|
|
<html lang='en'><head><meta charset='utf-8' /><meta name='pinterest' content='nopin' /><link href='../../../../static/css/style.css' rel='stylesheet' type='text/css' /><link href='../../../../static/css/print.css' rel='stylesheet' type='text/css' media='print' /><title>Deploying with Fabric & Mercurial / Steve Losh</title></head><body><header><a id='logo' href='https://stevelosh.com/'>Steve Losh</a><nav><a href='../../../index.html'>Blog</a> - <a href='https://stevelosh.com/projects/'>Projects</a> - <a href='https://stevelosh.com/photography/'>Photography</a> - <a href='https://stevelosh.com/links/'>Links</a> - <a href='https://stevelosh.com/rss.xml'>Feed</a></nav></header><hr class='main-separator' /><main id='page-blog-entry'><article><h1><a href='index.html'>Deploying with Fabric & Mercurial</a></h1><p class='date'>Posted on January 15th, 2009.</p><p>Earlier tonight I added support for the <a href="http://haveamint.com">Mint</a> <a href="http://haveamint.com/peppermill/pepper/11/bird_feeder/">Bird Feeder</a> plugin to my
|
||
|
|
site's <a href="https://stevelosh.com/rss/">RSS feeds</a>. Bird Feeder isn't designed to work with <a href="http://djangoproject.com/">Django</a> so I
|
||
|
|
had to change a few things to get it up and running. I mostly followed the
|
||
|
|
<a href="http://hicks-wright.net/blog/minty-django-feeds/">instructions on
|
||
|
|
Hicks-Wright.net</a> with a few
|
||
|
|
tweaks.</p>
|
||
|
|
|
||
|
|
<p>While I was trying to get things running smoothly I had to redeploy the site a
|
||
|
|
bunch of times so I could test the changes I made. If I were simply FTP'ing
|
||
|
|
the files over each time I wanted to redeploy it would have been a huge pain.
|
||
|
|
Fortunately, I have another way to do it that cuts down on my typing.</p>
|
||
|
|
|
||
|
|
<ol class="table-of-contents"><li><a href="index.html#s1-my-basic-deployment-steps">My Basic Deployment Steps</a></li><li><a href="index.html#s2-putting-bitbucket-in-the-middle">Putting BitBucket in the Middle</a><ol><li><a href="index.html#s3-set-up-your-local-machine">Set Up Your Local Machine</a></li><li><a href="index.html#s4-set-up-your-server">Set Up Your Server</a></li></ol></li><li><a href="index.html#s5-weaving-it-all-together-with-fabric">Weaving it All Together with Fabric</a><ol><li><a href="index.html#s6-my-current-setup">My Current Setup</a></li><li><a href="index.html#s7-extending-it">Extending It</a></li></ol></li><li><a href="index.html#s8-why-should-you-try-this">Why Should You Try This?</a></li><li><a href="index.html#s9-update">Update</a></li></ol>
|
||
|
|
|
||
|
|
<h2 id="s1-my-basic-deployment-steps"><a href="index.html#s1-my-basic-deployment-steps">My Basic Deployment Steps</a></h2>
|
||
|
|
|
||
|
|
<p>The code for my site is stored in a Mercurial repository. There are actually
|
||
|
|
three copies of the repository that I use:</p>
|
||
|
|
|
||
|
|
<ul>
|
||
|
|
<li>One is on my local machine, which I commit to as I make changes.</li>
|
||
|
|
<li>One is on <a href="http://bitbucket.org">BitBucket</a>, which I use to share the code with the public.</li>
|
||
|
|
<li>One is on my host's webserver, and is what actually gets served as the website.</li>
|
||
|
|
</ul>
|
||
|
|
|
||
|
|
<p>When I'm ready to deploy a new version of the site, I push the changes I've
|
||
|
|
made on my local repository to the one on BitBucket, then pull the changes
|
||
|
|
from BitBucket down to the server. Using push/pull means I don't have to worry
|
||
|
|
about transferring the files myself. Using BitBucket in the middle (instead of
|
||
|
|
going right from my local machine to the server) means I don't have to worry
|
||
|
|
about serving either repository myself and dealing with port forwarding or
|
||
|
|
security issues.</p>
|
||
|
|
|
||
|
|
<h2 id="s2-putting-bitbucket-in-the-middle"><a href="index.html#s2-putting-bitbucket-in-the-middle">Putting BitBucket in the Middle</a></h2>
|
||
|
|
|
||
|
|
<p>Using BitBucket as an intermediate repository is actually fairly simple. Here
|
||
|
|
are the basic steps I use to get a project up and running like this.</p>
|
||
|
|
|
||
|
|
<p>First, create a new repository on BitBucket. Make sure the name is what you
|
||
|
|
want. Feel free to make it private if you just want it for this, or public if
|
||
|
|
you want to <a href="../../../entry/2009/1/13/going-open-source/index.html">go open source</a>.</p>
|
||
|
|
|
||
|
|
<h3 id="s3-set-up-your-local-machine"><a href="index.html#s3-set-up-your-local-machine">Set Up Your Local Machine</a></h3>
|
||
|
|
|
||
|
|
<p>Clone this new, empty repository to your local machine. If you already have
|
||
|
|
code written you'll need to copy it into this folder after cloning. I don't
|
||
|
|
know if there's a way to push a brand-new repository to BitBucket; if you know
|
||
|
|
how please tell me.</p>
|
||
|
|
|
||
|
|
<p>On your local machine, edit the <code>.hg/hgrc</code> file in the repository and change
|
||
|
|
the default path to:</p>
|
||
|
|
|
||
|
|
<pre><code>default = ssh://hg@bitbucket.org/username/repositoryname/</code></pre>
|
||
|
|
|
||
|
|
<p>That will let you push/pull to and from BitBucket over SSH. Doing it that way
|
||
|
|
means you can use public/private key authentication and avoid typing your
|
||
|
|
password every single time. A fairly comprehensive guide to that can be found
|
||
|
|
<a href="http://www.securityfocus.com/infocus/1810">here</a>. You can ignore the
|
||
|
|
server-side configuration; you just need to add your public key on BitBucket's
|
||
|
|
account settings page and you should be set.</p>
|
||
|
|
|
||
|
|
<p><strong>UPDATE:</strong> I didn't realize it before, but BitBucket has a <a href="http://bitbucket.org/help/using-ssh/">guide to using SSH with BitBucket</a>. It's definitely worth looking at.</p>
|
||
|
|
|
||
|
|
<p>Now you should be able to use <code>hg push</code> and <code>hg pull</code> on your local machine to
|
||
|
|
push and pull changes to and from the BitBucket repository. The next step is
|
||
|
|
getting it set up on the server side.</p>
|
||
|
|
|
||
|
|
<h3 id="s4-set-up-your-server"><a href="index.html#s4-set-up-your-server">Set Up Your Server</a></h3>
|
||
|
|
|
||
|
|
<p>On your server, use <code>hg clone</code> to clone the BitBucket repository to wherever
|
||
|
|
you want to serve it from. Edit the <code>.hg/hgrc</code> file in that one and change the
|
||
|
|
default path to the same value as before:</p>
|
||
|
|
|
||
|
|
<pre><code>default = ssh://hg@bitbucket.org/username/repositoryname/</code></pre>
|
||
|
|
|
||
|
|
<p>Once again, set up public/private key authentication; this time between the
|
||
|
|
server and BitBucket. You can either copy your public and private keys from
|
||
|
|
your local machine to the server (if you trust/own it) or you can create a new
|
||
|
|
pair and add its public key to your BitBucket account as well.</p>
|
||
|
|
|
||
|
|
<p>While you're at it, set up public/private key authentication to go from your
|
||
|
|
local machine to your server too. It'll pay off in the long run.</p>
|
||
|
|
|
||
|
|
<p>Now that you've got both sides working, you can develop and deploy like so:</p>
|
||
|
|
|
||
|
|
<ul>
|
||
|
|
<li>Make changes on your local machine, committing as you go.</li>
|
||
|
|
<li>Push the changes to BitBucket.</li>
|
||
|
|
<li>SSH into your server and pull the changes down.</li>
|
||
|
|
<li>Restart the web server process if necessary.</li>
|
||
|
|
</ul>
|
||
|
|
|
||
|
|
<p>Not too bad! Instead of manually managing file transfers you can let Mercurial
|
||
|
|
do it for you. It'll only pull down the files that have changed, and will
|
||
|
|
always put them in exactly the right spot. That's pretty convenient, but we
|
||
|
|
can do better.</p>
|
||
|
|
|
||
|
|
<h2 id="s5-weaving-it-all-together-with-fabric"><a href="index.html#s5-weaving-it-all-together-with-fabric">Weaving it All Together with Fabric</a></h2>
|
||
|
|
|
||
|
|
<p>Being able to push and pull is all well and good, but that's still a lot of
|
||
|
|
typing. You need to enter a command to push your changes, a command to SSH to
|
||
|
|
the server, a command to change to the deploy directory, a command to pull the
|
||
|
|
changes, and a command to restart the server process. That's five commands,
|
||
|
|
which is four commands too many for me.</p>
|
||
|
|
|
||
|
|
<p>To automate the process, I use the wonderful <a href="http://www.fabfile.org/">Fabric</a> tool. If you haven't
|
||
|
|
seen it before you should take a look. To follow along with the rest of the
|
||
|
|
section you should read the examples on the site and install Fabric on your
|
||
|
|
local machine.</p>
|
||
|
|
|
||
|
|
<h3 id="s6-my-current-setup"><a href="index.html#s6-my-current-setup">My Current Setup</a></h3>
|
||
|
|
|
||
|
|
<p>Here's the fabfile I use for deploying my site to my host (<a href="http://www.webfaction.com/">WebFaction</a>).
|
||
|
|
It's pretty specific to my needs but I'm sure it will give you an idea of
|
||
|
|
where to start.</p>
|
||
|
|
|
||
|
|
<pre><code><span class="code"><span class="special">def</span><span class="keyword"> prod</span><span class="paren1">(<span class="code"></span>)</span>:
|
||
|
|
<span class="string">""</span><span class="string">"Set the target to production."</span><span class="string">""</span>
|
||
|
|
set<span class="paren1">(<span class="code">fab_hosts=<span class="paren2">[<span class="code"><span class="string">'sjl.webfactional.com'</span></span>]</span></span>)</span>
|
||
|
|
set<span class="paren1">(<span class="code">fab_key_filename=<span class="string">'/Users/sjl/.ssh/stevelosh'</span></span>)</span>
|
||
|
|
set<span class="paren1">(<span class="code">remote_app_dir=<span class="string">'~/webapps/stevelosh/stevelosh'</span></span>)</span>
|
||
|
|
set<span class="paren1">(<span class="code">remote_apache_dir=<span class="string">'~/webapps/stevelosh/apache2'</span></span>)</span>
|
||
|
|
|
||
|
|
<span class="special">def</span><span class="keyword"> deploy</span><span class="paren1">(<span class="code"></span>)</span>:
|
||
|
|
<span class="string">""</span><span class="string">"Deploy the site."</span><span class="string">""</span>
|
||
|
|
require<span class="paren1">(<span class="code"><span class="string">'fab_hosts'</span>, provided_by = <span class="paren2">[<span class="code">prod,</span>]</span></span>)</span>
|
||
|
|
local<span class="paren1">(<span class="code"><span class="string">"hg push"</span></span>)</span>
|
||
|
|
run<span class="paren1">(<span class="code"><span class="string">"cd $(remote_app_dir); hg pull; hg update"</span></span>)</span>
|
||
|
|
run<span class="paren1">(<span class="code"><span class="string">"cd $(remote_app_dir); python2.5 manage.py syncdb"</span></span>)</span>
|
||
|
|
run<span class="paren1">(<span class="code"><span class="string">"$(remote_apache_dir)/bin/stop; sleep 1; $(remote_apache_dir)/bin/start"</span></span>)</span>
|
||
|
|
|
||
|
|
<span class="special">def</span><span class="keyword"> debugon</span><span class="paren1">(<span class="code"></span>)</span>:
|
||
|
|
<span class="string">""</span><span class="string">"Turn debug mode on for the production server."</span><span class="string">""</span>
|
||
|
|
require<span class="paren1">(<span class="code"><span class="string">'fab_hosts'</span>, provided_by = <span class="paren2">[<span class="code">prod,</span>]</span></span>)</span>
|
||
|
|
run<span class="paren1">(<span class="code"><span class="string">"cd $(remote_app_dir); sed -i -e 's/DEBUG = .*/DEBUG = True/' deploy.py"</span></span>)</span>
|
||
|
|
run<span class="paren1">(<span class="code"><span class="string">"$(remote_apache_dir)/bin/stop; sleep 1; $(remote_apache_dir)/bin/start"</span></span>)</span>
|
||
|
|
|
||
|
|
<span class="special">def</span><span class="keyword"> debugoff</span><span class="paren1">(<span class="code"></span>)</span>:
|
||
|
|
<span class="string">""</span><span class="string">"Turn debug mode off for the production server."</span><span class="string">""</span>
|
||
|
|
require<span class="paren1">(<span class="code"><span class="string">'fab_hosts'</span>, provided_by = <span class="paren2">[<span class="code">prod,</span>]</span></span>)</span>
|
||
|
|
run<span class="paren1">(<span class="code"><span class="string">"cd $(remote_app_dir); sed -i -e 's/DEBUG = .*/DEBUG = False/' deploy.py"</span></span>)</span>
|
||
|
|
run<span class="paren1">(<span class="code"><span class="string">"$(remote_apache_dir)/bin/stop; sleep 1; $(remote_apache_dir)/bin/start"</span></span>)</span></span></code></pre>
|
||
|
|
|
||
|
|
<p>When I'm finished committing to my local repository and I want to deploy the
|
||
|
|
site, I just use the command <code>fab prod deploy</code> on my local machine. Fabric
|
||
|
|
pushes my local repository to BitBucket, logs into the server (with my public
|
||
|
|
key—no typing in passwords), pulls down the new changes from BitBucket
|
||
|
|
and restarts the server process.</p>
|
||
|
|
|
||
|
|
<p>I also set up a couple of debug commands so I can type <code>fab prod debugon</code> and
|
||
|
|
<code>fab prod debugoff</code> to change the <code>settings.DEBUG</code> option of my Django app.
|
||
|
|
Sometimes it's useful to turn on debug to find out exactly why a page is
|
||
|
|
breaking on the server.</p>
|
||
|
|
|
||
|
|
<h3 id="s7-extending-it"><a href="index.html#s7-extending-it">Extending It</a></h3>
|
||
|
|
|
||
|
|
<p>The reason I split off the <code>prod</code> command is so I can set up a separate test
|
||
|
|
app (on the server) in the future and reuse the <code>deploy</code> and <code>debug</code> commands.
|
||
|
|
All I'd need to do is add a <code>test</code> command, which might look something like
|
||
|
|
this:</p>
|
||
|
|
|
||
|
|
<pre><code><span class="code"><span class="special">def</span><span class="keyword"> test</span><span class="paren1">(<span class="code"></span>)</span>:
|
||
|
|
<span class="string">""</span><span class="string">"Set the target to test."</span><span class="string">""</span>
|
||
|
|
set<span class="paren1">(<span class="code">fab_hosts=<span class="paren2">[<span class="code"><span class="string">'sjl.webfactional.com'</span></span>]</span></span>)</span>
|
||
|
|
set<span class="paren1">(<span class="code">fab_key_filename=<span class="string">'/Users/sjl/.ssh/stevelosh'</span></span>)</span>
|
||
|
|
set<span class="paren1">(<span class="code">remote_app_dir = <span class="string">'~/webapps/stevelosh-test/stevelosh'</span></span>)</span>
|
||
|
|
set<span class="paren1">(<span class="code">remote_apache_dir = <span class="string">'~/webapps/stevelosh-test/apache2'</span></span>)</span></span></code></pre>
|
||
|
|
|
||
|
|
<p>Deploying the test site would then be a simple <code>fab test deploy</code> command.</p>
|
||
|
|
|
||
|
|
<h2 id="s8-why-should-you-try-this"><a href="index.html#s8-why-should-you-try-this">Why Should You Try This?</a></h2>
|
||
|
|
|
||
|
|
<p>After you've gotten all of this set up the first time it will start saving you
|
||
|
|
time every time you deploy. It also prevents stupid mistakes like FTP'ing your
|
||
|
|
files to the wrong directory on the server. It frees you from those headaches
|
||
|
|
and lets you concentrate on the real work to be done instead of the busywork.
|
||
|
|
Plus setting it up again for a second project is a breeze after you've done it
|
||
|
|
once.</p>
|
||
|
|
|
||
|
|
<p>Obviously the exact details of this won't be a perfect fit for everyone. Maybe
|
||
|
|
you prefer using git and <a href="http://github.com/">GitHub</a> for version control. I'm sure there's a
|
||
|
|
similar way to automate the process on that side of the fence. If you decide
|
||
|
|
to write something about the details of that please let me know and I'll link
|
||
|
|
it here.</p>
|
||
|
|
|
||
|
|
<p>My hope is that this will at least give you some ideas about saving yourself
|
||
|
|
some time. If that helps you create better websites, I'll be happy. Please
|
||
|
|
feel free to find me on <a href="http://twitter.com/stevelosh/">Twitter</a> with any questions or thoughts you have!</p>
|
||
|
|
|
||
|
|
<h2 id="s9-update"><a href="index.html#s9-update">Update</a></h2>
|
||
|
|
|
||
|
|
<p>It's been over a year since I originally wrote this entry and I've learned
|
||
|
|
quite a bit since then. I don't have the time right now to go back and rewrite
|
||
|
|
the entire article, but here are some of the main things that have changed:</p>
|
||
|
|
|
||
|
|
<ul>
|
||
|
|
<li>I now use <a href="http://clemesha.org/blog/2009/jul/05/modern-python-hacker-tools-virtualenv-fabric-pip/">virtualenv and pip</a> when deploying Django sites. It
|
||
|
|
sandboxes each site very nicely and makes it easy to deploy.</li>
|
||
|
|
</ul>
|
||
|
|
|
||
|
|
<ul>
|
||
|
|
<li><p>I no longer go "through" BitBucket when deploying — I push directly from my
|
||
|
|
local machine to the server. This eliminates an extra step, and I can always
|
||
|
|
push to BitBucket once I'm done with a series of changes.</p></li>
|
||
|
|
<li><p>Fabric is completely different. "Ownership/maintenance" of the project has
|
||
|
|
transferred to a new person and the format of fabfiles has drastically
|
||
|
|
changed.</p></li>
|
||
|
|
</ul>
|
||
|
|
|
||
|
|
<p>Here's a sample fabfile from one of the projects I'm working on now. There are
|
||
|
|
no comments, but I think most things should be self-explanitory. If you have
|
||
|
|
any questions just find me on <a href="http://twitter.com/stevelosh/">Twitter</a> and I'll be glad to answer them.</p>
|
||
|
|
|
||
|
|
<pre><code>from fabric.api import *
|
||
|
|
|
||
|
|
REMOTE_HG_PATH = '/home/sjl/bin/hg'
|
||
|
|
|
||
|
|
def prod():
|
||
|
|
"""Set the target to production."""
|
||
|
|
env.hosts = ['sjl.webfactional.com']
|
||
|
|
env.remote_app_dir = 'webapps/SAMPLE/SAMPLE'
|
||
|
|
env.remote_apache_dir = '~/webapps/SAMPLE/apache2'
|
||
|
|
env.local_settings_file = 'local_settings-prod.py'
|
||
|
|
env.remote_push_dest = 'ssh://webf/%s' % env.remote_app_dir
|
||
|
|
env.tag = 'production'
|
||
|
|
env.venv_name = 'SAMPLE_prod'
|
||
|
|
|
||
|
|
def test():
|
||
|
|
"""Set the target to test."""
|
||
|
|
env.hosts = ['sjl.webfactional.com']
|
||
|
|
env.remote_app_dir = 'webapps/SAMPLE_test/lindyhub'
|
||
|
|
env.remote_apache_dir = '~/webapps/SAMPLE_test/apache2'
|
||
|
|
env.local_settings_file = 'local_settings-test.py'
|
||
|
|
env.remote_push_dest = 'ssh://webf/%s' % env.remote_app_dir
|
||
|
|
env.tag = 'test'
|
||
|
|
env.venv_name = 'SAMPLE_test'
|
||
|
|
|
||
|
|
def deploy():
|
||
|
|
"""Deploy the site.
|
||
|
|
|
||
|
|
This will also add a local Mercurial tag with the name of the environment
|
||
|
|
(test or production) to your the local repository, and then sync it to
|
||
|
|
the remote repo as well.
|
||
|
|
|
||
|
|
This is nice because you can easily see which changeset is currently
|
||
|
|
deployed to a particular environment in 'hg glog'.
|
||
|
|
"""
|
||
|
|
require('hosts', provided_by=[prod, test])
|
||
|
|
require('remote_app_dir', provided_by=[prod, test])
|
||
|
|
require('remote_apache_dir', provided_by=[prod, test])
|
||
|
|
require('local_settings_file', provided_by=[prod, test])
|
||
|
|
require('remote_push_dest', provided_by=[prod, test])
|
||
|
|
require('tag', provided_by=[prod, test])
|
||
|
|
require('venv_name', provided_by=[prod, test])
|
||
|
|
|
||
|
|
local("hg tag --local --force %s" % env.tag)
|
||
|
|
local("hg push %s --remotecmd %s" % (env.remote_push_dest, REMOTE_HG_PATH))
|
||
|
|
put(".hg/localtags", "%s/.hg/localtags" % env.remote_app_dir)
|
||
|
|
run("cd %s; hg update -C %s" % (env.remote_app_dir, env.tag))
|
||
|
|
put("%s" % env.local_settings_file, "%s/local_settings.py" % env.remote_app_dir)
|
||
|
|
|
||
|
|
run("workon %s; cd %s; python manage.py syncdb" % (env.venv_name, env.remote_app_dir))
|
||
|
|
restart()
|
||
|
|
|
||
|
|
def restart():
|
||
|
|
"""Restart apache on the server."""
|
||
|
|
require('hosts', provided_by=[prod, test])
|
||
|
|
require('remote_apache_dir', provided_by=[prod, test])
|
||
|
|
|
||
|
|
run("%s/bin/stop; sleep 2; %s/bin/start" % (env.remote_apache_dir, env.remote_apache_dir))
|
||
|
|
|
||
|
|
def debugon():
|
||
|
|
"""Turn debug mode on for the server."""
|
||
|
|
require('hosts', provided_by=[prod, test])
|
||
|
|
require('remote_app_dir', provided_by=[prod, test])
|
||
|
|
require('remote_apache_dir', provided_by=[prod, test])
|
||
|
|
|
||
|
|
run("cd %s; sed -i -e 's/DEBUG = .*/DEBUG = True/' local_settings.py" % env.remote_app_dir)
|
||
|
|
restart()
|
||
|
|
|
||
|
|
def debugoff():
|
||
|
|
"""Turn debug mode off for the server."""
|
||
|
|
require('hosts', provided_by=[prod, test])
|
||
|
|
require('remote_app_dir', provided_by=[prod, test])
|
||
|
|
require('remote_apache_dir', provided_by=[prod, test])
|
||
|
|
|
||
|
|
run("cd %s; sed -i -e 's/DEBUG = .*/DEBUG = False/' local_settings.py" % env.remote_app_dir)
|
||
|
|
restart()
|
||
|
|
</code></pre>
|
||
|
|
</article></main><hr class='main-separator' /><footer><nav><a href='https://github.com/sjl/'>GitHub</a> ・ <a href='https://twitter.com/stevelosh/'>Twitter</a> ・ <a href='https://instagram.com/thirtytwobirds/'>Instagram</a> ・ <a href='https://hg.stevelosh.com/.plan/'>.plan</a></nav></footer></body></html>
|