marcuskammer/emacs.d
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
emacs.d/clones/lisp/stevelosh.com/blog/2010/01/moving-from-django-to-hyde/index.html

395 lines
23 KiB
HTML
Raw Normal View History

Update clones
2022-10-07 15:47:14 +02:00
<!DOCTYPE html>
<html lang='en'><head><meta charset='utf-8' /><meta name='pinterest' content='nopin' /><link href='../../../../static/css/style.css' rel='stylesheet' type='text/css' /><link href='../../../../static/css/print.css' rel='stylesheet' type='text/css' media='print' /><title>Moving from Django to Hyde / Steve Losh</title></head><body><header><a id='logo' href='https://stevelosh.com/'>Steve Losh</a><nav><a href='../../../index.html'>Blog</a> - <a href='https://stevelosh.com/projects/'>Projects</a> - <a href='https://stevelosh.com/photography/'>Photography</a> - <a href='https://stevelosh.com/links/'>Links</a> - <a href='https://stevelosh.com/rss.xml'>Feed</a></nav></header><hr class='main-separator' /><main id='page-blog-entry'><article><h1><a href='index.html'>Moving from Django to Hyde</a></h1><p class='date'>Posted on January 15th, 2010.</p><p>Almost exactly one year ago I posted a blog entry about how I <a href="../../../2009/01/site-redesign/index.html">rewrote this
site</a> using <a href="http://djangoproject.com/">Django</a>. It's a new year, and once again I've
rewritten the site.</p>
<p>If you've visited my site before you may have noticed some small style tweaks.
I've cleaned up a few things and I think the site looks better overall. The
biggest change, however, is that I completely rewrote the core — this time
with <a href="http://hyde.github.io/">Hyde</a>.</p>
<p>Hyde is a &quot;static site generator&quot; written in <a href="http://python.org/">Python</a>, similar to
<a href="http://jekyllrb.com/">Jekyll</a>, <a href="https://bitbucket.org/jek/blatter">Blatter</a> and <a href="http://inky.github.com/pilcrow/">Pilcrow</a>. I chose it over the others because
it's written in Python (unlike Jekyll), more flexible than Blatter and more
mature than Pilcrow.</p>
<p>Rewriting an existing site that gets a decent amount of visitors is different
than creating a new site from scratch, so I decided to write this entry to
describe some of the issues I ran into and how I tackled them.</p>
<ol class="table-of-contents"><li><a href="index.html#s1-why-static">Why Static?</a><ol><li><a href="index.html#s2-less-memory-needed">Less Memory Needed</a></li><li><a href="index.html#s3-static-pages-are-faster">Static Pages are Faster</a></li><li><a href="index.html#s4-easier-to-maintain">Easier to Maintain</a></li><li><a href="index.html#s5-easier-to-backup">Easier to Backup</a></li></ol></li><li><a href="index.html#s6-getting-started">Getting Started</a></li><li><a href="index.html#s7-layout-and-styling">Layout and Styling</a><ol><li><a href="index.html#s8-cleanup">Cleanup</a></li><li><a href="index.html#s9-rewriting-the-layout-and-styles">Rewriting the Layout and Styles</a></li><li><a href="index.html#s10-a-new-css-framework">A New CSS Framework</a></li><li><a href="index.html#s11-pagination">Pagination</a></li></ol></li><li><a href="index.html#s12-using-fabric-to-type-less">Using Fabric to Type Less</a></li><li><a href="index.html#s13-moving-the-content">Moving the Content</a></li><li><a href="index.html#s14-converting-the-comments">Converting the Comments</a></li><li><a href="index.html#s15-rewriting-the-old-urls">Rewriting the Old URLs</a></li><li><a href="index.html#s16-creating-an-rss-feed">Creating an RSS Feed</a></li><li><a href="index.html#s17-merging-the-repositories">Merging the Repositories</a></li><li><a href="index.html#s18-still-to-come">Still to Come</a><ol><li><a href="index.html#s19-ago-dates">&quot;Ago&quot; Dates</a></li><li><a href="index.html#s20-categories">Categories</a></li><li><a href="index.html#s21-use-lesscss">Use LessCSS</a></li></ol></li><li><a href="index.html#s22-view-the-code">View the Code</a></li></ol>
<h2 id="s1-why-static"><a href="index.html#s1-why-static">Why Static?</a></h2>
<p>There are a couple of reasons I decided to switch to a static site.</p>
<h3 id="s2-less-memory-needed"><a href="index.html#s2-less-memory-needed">Less Memory Needed</a></h3>
<p>I use <a href="http://www.webfaction.com/">WebFaction</a> for my web hosting, and with my current plan I have a
limit on how much memory I can use at any given time. Each Django site takes
up around 20mb of memory on the server.</p>
<p>By switching to a static set of HTML pages for my site I save those 20mb so I
can use them for something else (like a staging site for another project). My
site doesn't particularly <em>need</em> all of the functionality Django can provide,
so I figured I'd switch to a static site and save the memory.</p>
<h3 id="s3-static-pages-are-faster"><a href="index.html#s3-static-pages-are-faster">Static Pages are Faster</a></h3>
<p>My site doesn't get an enormous amount of traffic, so the Django instances
weren't really working very hard. Still, serving a static page through Apache
or nginx is always going to be faster than running it through Django.
Faster-loading pages is always a good thing, even if the speed increase isn't
very large.</p>
<h3 id="s4-easier-to-maintain"><a href="index.html#s4-easier-to-maintain">Easier to Maintain</a></h3>
<p>This is the main reason I switched. Previously, to edit a blog entry I would
log in through Django's admin interface and edit the entry. With Hyde, I
simply edit a file (or create a new one) on my machine and then run a single
command to publish it.</p>
<p>Another problem with the old site is that I needed to be connected to the
internet to get to the admin interface, so I couldn't update entries (or
publish new ones) offline. I usually have an internet connection, but
occasionally I don't. Now I can edit as much as I like on my own machine and
only need a connection to publish the finished product.</p>
<h3 id="s5-easier-to-backup"><a href="index.html#s5-easier-to-backup">Easier to Backup</a></h3>
<p>One more advantage of Hyde is that the site structure <em>and</em> content are all
held in the same place. I keep everything in a <a href="http://mercurial-scm.org/">Mercurial</a> repository, and
so every time I push that repository somewhere it creates a full backup of the
site's code <em>and</em> content. If WebFaction's server catches on fire I still have
everything on my local machine (and on BitBucket).</p>
<p>I've toyed around with backing up Django's database tables when I had my old
site, but this new method is far less work.</p>
<h2 id="s6-getting-started"><a href="index.html#s6-getting-started">Getting Started</a></h2>
<p>I wanted a fresh start when I was rewriting the site, so I went ahead and
created a brand new folder and Mercurial repository for it.</p>
<p>I used <code>hyde --init</code> to lay out a skeleton in the new folder. Then I stripped
out a lot of the default items that get added with <code>--init</code> and created the
directory structure I wanted to use, with stubs for each of the main content
files I knew I'd need.</p>
<p>Finally, I went ahead and filled some of the basic values in the <code>settings.py</code>
file.</p>
<h2 id="s7-layout-and-styling"><a href="index.html#s7-layout-and-styling">Layout and Styling</a></h2>
<p>Once I had a skeleton in place I started working on layout of the site.</p>
<h3 id="s8-cleanup"><a href="index.html#s8-cleanup">Cleanup</a></h3>
<p>The templates created by <code>hyde --init</code> are functional, but when you look at
the code they're a mess. The indentation is strange and inconsistent and
there's trailing whitespace all over the place. I like clean code so I sat
down and cleaned everything up before I started making any real changes.</p>
<h3 id="s9-rewriting-the-layout-and-styles"><a href="index.html#s9-rewriting-the-layout-and-styles">Rewriting the Layout and Styles</a></h3>
<p>After I finished cleaning up the templates I duplicated the HTML structure and
styles of the old site from scratch. The old site had gone through a bunch of
iterations and I was a bit sloppy in my editing, so there was a lot of cruft
that had snuck in. I wanted a truly <em>fresh</em> start for the site, so I buckled
down and did it all again.</p>
<h3 id="s10-a-new-css-framework"><a href="index.html#s10-a-new-css-framework">A New CSS Framework</a></h3>
<p>Previously I used the <a href="http://www.blueprintcss.org/">Blueprint CSS framework</a> to make laying out
the site easier. Blueprint is a great framework, but it's more powerful than I
need for a site as simple as this one. The site now uses <a href="https://gist.github.com/btbytes/437704">Aardvark Legs</a>,
which is a much simpler framework that simply sets up a great vertical rhythm
and leaves you free to lay out the horizontal structure yourself.</p>
<h3 id="s11-pagination"><a href="index.html#s11-pagination">Pagination</a></h3>
<p>Before the rewrite the list of blog entries used to be paginated. Hyde
supports pagination but I decided against using it because I simply don't
write enough to make it necessary. All the blog entries are now listed on a
single page, which means that you can use Cmd+F to find an article if you're
looking for something specific.</p>
<h2 id="s12-using-fabric-to-type-less"><a href="index.html#s12-using-fabric-to-type-less">Using Fabric to Type Less</a></h2>
<p>I realized very quickly that typing <code>hyde -g -s . &amp;&amp; hyde -w -s .</code> would get
old pretty quickly, so I installed <a href="http://www.fabfile.org/">Fabric</a> and wrote a fabfile.</p>
<p>Fabric is a tool written in Python that lets you define tasks and execute them
by running <code>fab taskname</code>. Fabfiles are pure Python, so you can build larger
tasks out of smaller ones very easily and do just about anything you want.
It's similar to <a href="http://ant.apache.org/">ant</a>, but without the excessive over-engineering.</p>
<p>You can view the <a href="http://bitbucket.org/sjl/stevelosh/src/tip/fabfile.py">current fabfile</a> on BitBucket. At the time of this
entry it looks like this:</p>
<pre><code><span class="code"><span class="symbol">from</span> fabric.api <span class="symbol">import</span> *
<span class="symbol">import</span> os
<span class="symbol">import</span> fabric.contrib.project as project
PROD = <span class="string">'sjl.webfactional.com'</span>
DEST_PATH = <span class="string">'/home/sjl/webapps/slc/'</span>
ROOT_PATH = os.path.abspath<span class="paren1">(<span class="code">os.path.dirname<span class="paren2">(<span class="code">__file__</span>)</span></span>)</span>
DEPLOY_PATH = os.path.join<span class="paren1">(<span class="code">ROOT_PATH, <span class="string">'deploy'</span></span>)</span>
<span class="special">def</span><span class="keyword"> clean</span><span class="paren1">(<span class="code"></span>)</span>:
local<span class="paren1">(<span class="code"><span class="string">'rm -rf ./deploy'</span></span>)</span>
<span class="special">def</span><span class="keyword"> regen</span><span class="paren1">(<span class="code"></span>)</span>:
clean<span class="paren1">(<span class="code"></span>)</span>
local<span class="paren1">(<span class="code"><span class="string">'hyde -g -s .'</span></span>)</span>
<span class="special">def</span><span class="keyword"> serve</span><span class="paren1">(<span class="code"></span>)</span>:
local<span class="paren1">(<span class="code"><span class="string">'hyde -w -s .'</span></span>)</span>
<span class="special">def</span><span class="keyword"> reserve</span><span class="paren1">(<span class="code"></span>)</span>:
regen<span class="paren1">(<span class="code"></span>)</span>
serve<span class="paren1">(<span class="code"></span>)</span>
<span class="special">def</span><span class="keyword"> smush</span><span class="paren1">(<span class="code"></span>)</span>:
local<span class="paren1">(<span class="code"><span class="string">'smusher ./media/images'</span></span>)</span>
<span class="symbol">@hosts</span><span class="paren1">(<span class="code">PROD</span>)</span>
<span class="special">def</span><span class="keyword"> publish</span><span class="paren1">(<span class="code"></span>)</span>:
regen<span class="paren1">(<span class="code"></span>)</span>
project.rsync_project<span class="paren1">(<span class="code">
remote_dir=DEST_PATH,
local_dir=DEPLOY_PATH.rstrip<span class="paren2">(<span class="code"><span class="string">'/'</span></span>)</span> + <span class="string">'/'</span>,
delete=True
</span>)</span></span></code></pre>
<p>The task I use most often while developing is <code>fab reserve</code>, which regenerates
the site and then starts serving it so I can view the result in a browser.</p>
<p>I use <code>fab smush</code> whenever I add new images. This runs <a href="http://github.com/grosser/smusher">smusher</a> on all of
the images to optimize them without reducing quality.</p>
<p>When I'm ready to publish changes to the live site I run <code>fab publish</code>, which
will regenerate my local version and copy it up to the WebFaction server.</p>
<h2 id="s13-moving-the-content"><a href="index.html#s13-moving-the-content">Moving the Content</a></h2>
<p>The content of the old site (blog entries, projects, and static pages like the
<a href="https://stevelosh.com/about/">about page</a>) was fairly easy to migrate over to the new one, because both
versions use <a href="https://daringfireball.net/projects/markdown/">Markdown</a> to format the text.</p>
<p>First I created an empty file for each page with a filename that matched the
&quot;slug&quot; (last part of the URL) of the old page. Then I manually copied over the
title, creation time and content for every page. I could have written a script
to do this, but I don't have enough pages on the site to make it worth the
time.</p>
<h2 id="s14-converting-the-comments"><a href="index.html#s14-converting-the-comments">Converting the Comments</a></h2>
<p>At this point the new site was looking very much like the old one. The styles
were (roughly) matching and the blog posts, entries, and static pages were all
rendered nicely.</p>
<p>The next big task was migrating all of the comments. Since the new site is
static it can't handle dynamic content like comments on its own, so I decided
to use <a href="http://disqus.com/">Disqus</a>. I use Disqus for the comments on the <a href="http://hgtip.com/">hg tip</a> site and
it's very nice. For that site I used it from the beginning, but for this
rewrite I needed to somehow import all the old comments.</p>
<p>To migrate the comments over I used <a href="http://github.com/arthurk/django-disqus">django-disqus</a>, but there was a small
snag that I needed to deal with first.</p>
<p>When I first wrote the old site I was just getting back into Django after not
using it for a long time. I didn't know about Django's built-in comment
models, so I created my own. They weren't as good as Django's, but they did
the job and I didn't care enough to change them.</p>
<p>This became a problem when it was time to import the old comments into Disqus.
django-disqus only supports importing comments from Django's built-in comment
models. To work around this I first had to convert the old comments into
Django's built-in ones. I wrote a <a href="http://bitbucket.org/sjl/stevelosh/src/da98105753a1/convert-comments.py">small, hacky Python
script</a> to do it:</p>
<pre><code>:::python
#!/usr/bin/env python
from django.core.management import setup_environ
import settings
setup_environ(settings)
from markdown import Markdown
from django.contrib.comments.models import Comment
from django.contrib.sites.models import Site
from stevelosh.blog.models import Comment as BlogComment
from stevelosh.projects.models import Comment as ProjectComment
mdown = Markdown()
site = Site.objects.all()[0]
blog_comments = BlogComment.objects.filter(spam=False)
project_comments = ProjectComment.objects.filter(spam=False)
for bc in blog_comments:
c = Comment()
c.content_object = bc.entry
c.user_name = bc.name
c.comment = mdown.convert(bc.body)
c.submit_date = bc.submitted
c.site = site
c.is_public = True
c.is_removed = False
c.save()
# print 'http://%s%s' % (site.domain, c.content_object.get_absolute_url())
for pc in project_comments:
c = Comment()
c.content_object = pc.project
c.user_name = pc.name
c.comment = mdown.convert(pc.body)
c.submit_date = pc.submitted
c.site = site
c.is_public = True
c.is_removed = False
c.save()
# print 'http://%s%s' % (site.domain, c.content_object.get_absolute_url())
</code></pre>
<p>Yes, it's ugly. No, I don't care. It was run once or twice locally and once on
the live server. It worked and I'll never need to run it again.</p>
<p>Once I had the comments converted I could use django-disqus to migrate them.
The import went very smoothly — I ran one command and after a couple of
minutes everything was in Disqus. Once that finished it was just a matter of
adding the Disqus JavaScript to one of the templates.</p>
<h2 id="s15-rewriting-the-old-urls"><a href="index.html#s15-rewriting-the-old-urls">Rewriting the Old URLs</a></h2>
<p>Since I was pretty much starting from scratch with this rewrite I decided to
clean up the URL structure of my blog. Previously a blog entry's URL looked
like this:</p>
<pre><code>http://stevelosh.com/blog/entry/2009/08/30/a-guide-to-branching-in-mercurial/</code></pre>
<p>The <code>/entry/</code> part of the URL was useless — it just took up space, so I got
rid of it. The year and month are useful to get an idea of how old an entry is
just by looking at the link, so I left them in. The day, however, probably
doesn't matter that much, so I took it out.</p>
<p>The new URLs look like this:</p>
<pre><code>http://stevelosh.com/blog/2009/08/a-guide-to-branching-in-mercurial/</code></pre>
<p>The problem with rewriting the URL structure is that there are already links
around the web pointing at my entries. I didn't want those old links to break,
so I crafted an <a href="http://bitbucket.org/sjl/stevelosh/src/tip/content/.htaccess"><code>.htaccess</code> file</a> that would rewrite the old URLs
into the new ones:</p>
<pre><code>{% templatetag openblock %} if GENERATE_CLEAN_URLS {% templatetag closeblock %}
RewriteEngine on
RewriteBase {% templatetag openvariable %} node.site.settings.SITE_ROOT {% templatetag closevariable %}
# Old URLs
RewriteRule ^blog/entry/(\d+)/(\d\d)/\d+/([^/]*)/?$ /blog/$1/$2/$3/ [R=301,L]
RewriteRule ^blog/entry/(\d+)/(\d)/\d+/([^/]*)/?$ /blog/$1/0$2/$3/ [R=301,L]
{% templatetag openblock %} hyde_listing_page_rewrite_rules {% templatetag closeblock %}
# listing pages whose names are the same as their enclosing folder's
RewriteCond %{REQUEST_FILENAME}/$1.html -f
RewriteRule ^([^/]*)/$ %{REQUEST_FILENAME}/$1.html
# regular pages
RewriteCond %{REQUEST_FILENAME}.html -f
RewriteRule ^.*$ %{REQUEST_FILENAME}.html
{% templatetag openblock %} endif {% templatetag closeblock %}
</code></pre>
<p>Most of that file is the stock Hyde <code>.htaccess</code> file — the two lines under <code>#
Old URLs</code> are the ones that I added. It took me a while to get it right
because I don't work with Apache's <code>mod_rewrite</code> very often, but it was worth
it to avoid breaking all of the old links.</p>
<h2 id="s16-creating-an-rss-feed"><a href="index.html#s16-creating-an-rss-feed">Creating an RSS Feed</a></h2>
<p>I had an RSS feed for the old site which Django made very easy to set up. I
definitely needed a feed for the new site, and fortunately Hyde provides a
simple sample template that can be used to make an ATOM feed.</p>
<p>I cleaned up the whitespace and formatting of that template a bit, adjusted a
few variables for my needs, put it on <a href="http://www.feedburner.com/">FeedBurner</a> and everything was ready.</p>
<h2 id="s17-merging-the-repositories"><a href="index.html#s17-merging-the-repositories">Merging the Repositories</a></h2>
<p>The last step before I was finished was to merge the old and new repositories
together so I wouldn't lose any of the history. It's probably not <em>too</em>
important to keep the old site's history around, but I put a lot of work into
it over the past year and it has some sentimental value.</p>
<p>Fortunately it's very easy to <a href="http://hgtip.com/tips/advanced/2009-11-17-combining-repositories/">combine Mercurial repositories</a>.
I just pulled the old repository into the new one, merged the old head into
the new one while discarding all the changes, and pushed it up to BitBucket.</p>
<h2 id="s18-still-to-come"><a href="index.html#s18-still-to-come">Still to Come</a></h2>
<p>At this point I felt the site was ready to be released, so I set up a new site
on WebFaction and used Fabric to deploy it.</p>
<p>I'm very happy with the result, but there are still a few things I'm going to
fix/change in the future.</p>
<h3 id="s19-ago-dates"><a href="index.html#s19-ago-dates">&quot;Ago&quot; Dates</a></h3>
<p><strong>UPDATE:</strong> This is done. I've started using <a href="http://timeago.yarp.com/">timeago.js</a> to render the
&quot;ago&quot; dates.</p>
<p>If you look at the top of each blog entry and project there's a line beneath
the title that looks something like:</p>
<pre><code>Posted on Monday, November 16, 2009 (1 month, 3 weeks ago).</code></pre>
<p>The <code>(1 month, 3 weeks ago)</code> part of that line is something that I really
appreciate on blogs. When they just list the date I always have to do the math
in my head to figure out roughly how old something is.</p>
<p>With a static site, however, those times will quickly become inaccurate if I
don't regenerate and publish the site regularly. I'm still thinking about the
best way to work this out.</p>
<p>One option is to use a cron job on the WebFaction server to regenerate the
site every day, which would keep the times <em>fairly</em> accurate.</p>
<p>Another option would be to use a bit of JavaScript to calculate and render the
time when the page is loaded. This would make it <em>completely</em> accurate but
wouldn't work if someone is browsing with JavaScript turned off.</p>
<h3 id="s20-categories"><a href="index.html#s20-categories">Categories</a></h3>
<p><strong>UPDATE:</strong> I've added categories. Check out <a href="http://bitbucket.org/sjl/stevelosh/changeset/08d7552b6237/">this changeset</a> to see what I had to do.</p>
<p>Right now all the blog entries (and projects) are listed in a single
chronological list. It would be great to break them up into categories so
people can easily find the articles they're interested in.</p>
<p>Hyde supports categories but I haven't spent the time to learn how to use them
yet. I also need to figure out a way to work a list of categories into the
design without cluttering things up too much.</p>
<h3 id="s21-use-lesscss"><a href="index.html#s21-use-lesscss">Use LessCSS</a></h3>
<p><strong>UPDATE:</strong> This is done. The main style file now uses LessCSS.</p>
<p><a href="http://lesscss.org/">LessCSS</a> is a language that extends CSS with some useful features like
variables, mixins, operations and nested rules. It can make the styles of a
site much, much cleaner.</p>
<p>Hyde includes a LessCSS &quot;processor&quot; that will automatically render your
LessCSS files into normal CSS. I'm planning on rewriting the site's styles in
LessCSS and using the processor once I get some free time.</p>
<h2 id="s22-view-the-code"><a href="index.html#s22-view-the-code">View the Code</a></h2>
<p>The code is on <a href="http://bitbucket.org/sjl/stevelosh/">BitBucket</a> and <a href="http://github.com/sjl/stevelosh/">GitHub</a>. Feel free to poke
around and see how I've set things up. If you have questions or suggestions
I'd love to hear them!</p>
</article></main><hr class='main-separator' /><footer><nav><a href='https://github.com/sjl/'>GitHub</a> ・ <a href='https://twitter.com/stevelosh/'>Twitter</a> ・ <a href='https://instagram.com/thirtytwobirds/'>Instagram</a> ・ <a href='https://hg.stevelosh.com/.plan/'>.plan</a></nav></footer></body></html>
Reference in a new issue Copy permalink