diff --git a/docs/hacking.md b/docs/hacking.md new file mode 100644 index 0000000..82ef0a1 --- /dev/null +++ b/docs/hacking.md @@ -0,0 +1,141 @@ +## Coleslaw: A Hacker's Guide + +Here we'll provide an overview of key concepts and technical decisions +in *coleslaw* and a few suggestions about future directions. Please +keep in mind that *coleslaw* was written on a lark when 3 friends had +the idea to each complete their half-dreamed wordpress replacement in +a week. Though it has evolved considerably since it's inception, like +any software some mess remains. + +## Core Concepts + +### Data and Deployment + +**Coleslaw** is pretty fundamentally tied to the idea of git as both a +backing data store and a deployment method (via `git push`). The +consequence is that you need a bare repo somewhere with a post-recieve +hook. That post-recieve hook +([example](https://github.com/redline6561/coleslaw/blob/master/examples/example.post-receive)) +will checkout the repo to a **$TMPDIR** and call `(coleslaw:main $TMPDIR)`. + +It is then coleslaw's job to load all of your content, your config and +templates, and render the content to disk. Deployment is done by +updating a symlink and the default install assumes your webserver will +be configured to serve from that symlink. However, there are plugins +for deploying to Heroku, S3, and Github Pages. + +### Blogs vs Sites + +**Coleslaw** is blogware. When I designed it, I only cared that it +could replace my server's wordpress install. As a result, the code is +still structured in terms of POSTs and INDEXes. Roughly speaking, a +POST is a blog entry and an INDEX is a collection of POSTs or other +content. An INDEX really only serves to group a set of content objects +on a page, it isn't content itself. + +This isn't ideal if you're looking for a full-on static site +generator. Content Types were added in 0.8 as a step towards making +*coleslaw* suitable for more use cases but still have some +limitations. Chiefly, the association between Content Types, their +template, and their inclusion in an INDEX is presently ad-hoc. + +### Templates and Theming + +User configs are allowed to specify a theme, otherwise the default is +used. A theme consists of a directory under "themes/" containing css, +images, and at least 3 templates: Base, Index, and Post. + +**Coleslaw** exclusively uses +[cl-closure-template](https://github.com/archimag/cl-closure-template) +for templating which is a well documented CL implementation of +Google's Closure Templates. Each template file should be in a +namespace like `coleslaw.theme.theme-name`. + +Each template creates a lisp function in the theme's package when +loaded. These functions take a property list (or plist) as an argument +and return rendered HTML. **Coleslaw** defines a helper called +`theme-fn` for easy access to the template functions. + +### The Lifecycle of a Page + +- `(load-content)` + +A page starts, obviously, with a file. When +*coleslaw* loads your content, it iterates over a list of content +types (i.e. subclasses of CONTENT). For each content type, it +iterates over all files in the repo with a matching extension, +e.g. ".post" for POSTs. Objects of the appropriate class are created +from each matching file and inserted into the `*content*` hash-table. + +- `(compile-blog dir)` + +Compilation starts by ensuring the staging directory (`/tmp/coleslaw/` +by default) exists, cd'ing there, and copying over any necessary theme +assets. Then *coleslaw* iterates over the content types, calling the +`publish` method on each one. Publish creates any non-INDEX pages for +the objects of that content type by iterating over the objects in an +appropriate fashion, rendering them, and passing the result to +`write-page` (which should probably just be renamed to `write-file`). + +After this, `render-indices` and `render-feeds` are called, and an +'index.html' symlink is created to point to the first reverse +chronological index. + +- `(deploy dir)` + +Finally, we move the staging directory to a timestamped path under the +the config's `:deploy-dir`, delete the directory pointed to by the old +'.prev' symlink, point '.curr' at '.prev', and point '.curr' at our +freshly built site. + +## Areas for Improvement + +### Better Content Types + +Creating a new content type should be both straightforward and doable +as a plugin. All that is really required is a subclass of CONTENT with +any needed slots, a template, a `render` method to call the template +with any needed options, a `page-url` method for layout, and a +`publish` method. + +Unfortunately, this does not solve: + +1. The issue of compiling the template at load-time and making sure it + was installed in the theme package. The plugin would need to do + this itself or the template would need to be included in 'core'. +2. More seriously, there is no formal relationship between content + types and indices. Indices include *ALL* objects in the `*content*` + hash table. This may be undesirable and doesn't permit indices + dedicated to particular content types. + +### Layouts and Paths + +Defining a page-url for every content-object and index seems a bit +silly. It also spreads information about the site layout throughout +the codebase, it might be better to have a slot in the config that +defines this information with a key to go with each format string. +Adding a new content-type as a plugin could then provide a default +by banging on the config or specify the path in its `enable` options. + +### Incremental Compilation + +Incremental compilation is doable, even straightforward if you ignore +indices. It is also preferable to building the site in parallel as +avoiding work is better than using more workers. Moreover, being +able to determine (and expose) what files just changed enables new +functionality such as plugins that cross-post to tumblr. + +Git's post-receieve hook is supposed to get a list of refs on $STDIN. +A brave soul could update our post-receive script to figure out the +original hash and pass that along to `coleslaw:main`. We could then +use it to run `git diff --name-status $HASH HEAD` to find changed +files and act accordingly. + +This is a cool project and the effects are far reaching. Among other +things the existing deployment model would not work as it involves +rebuilding the entire site. In all likelihood we would want to update +the site 'in-place'. Atomicity of filesystem operations would be a +reasonable concern. Also, every numbered INDEX would have to be +regenerated along with any tag or month indices matching the +modified files. If incremental compilation is a goal, simply +disabling the indices may be appropriate for certain users. diff --git a/docs/plugin-api.md b/docs/plugin-api.md index 5b8c08c..a6c5cab 100644 --- a/docs/plugin-api.md +++ b/docs/plugin-api.md @@ -15,12 +15,42 @@ # Extension Points -* **New functionality via JS**, for example the Disqus and Mathjax plugins. In this case, the plugin's `enable` function should call [`add-injection`](http://redlinernotes.com/docs/coleslaw.html#add-injection_func) with an injection and a keyword. The injection itself is a list of the string to insert and a lambda or function that can be called on a content instance to determine whether the injection should be included on the page. The keyword specifies whether the injected text goes in the HEAD or BODY element. The [Disqus plugin](http://github.com/redline6561/coleslaw/blob/master/plugins/disqus.lisp) is a good example of this. +* **New functionality via JS**, for example the Disqus and Mathjax + plugins. In this case, the plugin's `enable` function should call + [`add-injection`](http://redlinernotes.com/docs/coleslaw.html#add-injection_func) + with an injection and a keyword. The injection itself is a list of + the string to insert and a lambda or function that can be called on + a content instance to determine whether the injection should be + included on the page. The keyword specifies whether the injected + text goes in the HEAD or BODY element. The + [Disqus plugin](http://github.com/redline6561/coleslaw/blob/master/plugins/disqus.lisp) + is a good example of this. -* **New markup formats**, for example the [ReStructuredText plugin](http://github.com/redline6561/coleslaw/blob/master/plugins/rst.lisp), can be created by definining an appropriate `render-content` method. The method takes `text` and `format` arguments and is [EQL-specialized](http://www.gigamonkeys.com/book/object-reorientation-generic-functions.html#defmethod) on the format. Format should be a keyword matching the file extension (or pathname-type) of the markup format. (eg. `:rst` for ReStructuredText) +* **New markup formats**, for example the + [ReStructuredText plugin](http://github.com/redline6561/coleslaw/blob/master/plugins/rst.lisp), + can be created by definining an appropriate `render-content` + method. The method takes `text` and `format` arguments and is + [EQL-specialized](http://www.gigamonkeys.com/book/object-reorientation-generic-functions.html#defmethod) + on the format. Format should be a keyword matching the file + extension (or pathname-type) of the markup format. (eg. `:rst` for + ReStructuredText) -* **New hosting options**, for example the [Amazon S3 plugin](http://github.com/redline6561/coleslaw/blob/master/plugins/s3.lisp), can be created by definining a `deploy :after` method. The method takes a staging directory, likely uninteresting in the `:after` stage. But by importing `*config*` from the coleslaw package and getting its deploy location with `(deploy-dir *config*)` a number of interesting options become possible. +* **New hosting options**, for example the + [Amazon S3 plugin](http://github.com/redline6561/coleslaw/blob/master/plugins/s3.lisp), + can be created by definining a `deploy :after` method. The method + takes a staging directory, likely uninteresting in the `:after` + stage. But by importing `*config*` from the coleslaw package and + getting its deploy location with `(deploy-dir *config*)` a number of + interesting options become possible. -* **New content types**, for example a bookmark-like content type such as ["shouts"](http://paste.lisp.org/display/134453). This has not yet been attempted as a plugin but should be possible without much difficulty. +* **New content types**, for example a bookmark-like content type such + as ["shouts"](http://paste.lisp.org/display/134453). This has not + yet been attempted as a plugin but should be possible without much + difficulty. -* **New service integrations**, for example crossposting to twitter/facebook/tumblr/livejournal/etc, should also be possible by adding an :after hook to the deploy method as with hosting. Alternately, an :after hook on the [`render`](http://redlinernotes.com/docs/coleslaw.html#render_func) method might be used. +* **New service integrations**, for example crossposting to + twitter/facebook/tumblr/livejournal/etc, should also be possible by + adding an :after hook to the deploy method as with + hosting. Alternately, an :after hook on the + [`render`](http://redlinernotes.com/docs/coleslaw.html#render_func) + method might be used. diff --git a/src/coleslaw.lisp b/src/coleslaw.lisp index e8e0c7e..632449c 100644 --- a/src/coleslaw.lisp +++ b/src/coleslaw.lisp @@ -78,13 +78,13 @@ Additional args to render CONTENT can be passed via RENDER-ARGS." (update-symlink curr new-build)))) (defun main (&optional config-key) - "Load the user's config file, then compile and deploy the blog." + "Load the user's config file, then compile and deploy the site." (load-config config-key) (load-content) (compile-theme (theme *config*)) - (let ((blog (staging-dir *config*))) - (compile-blog blog) - (deploy blog))) + (let ((dir (staging-dir *config*))) + (compile-blog dir) + (deploy dir))) (defun preview (path &optional (content-type 'post)) "Render the content at PATH under user's configured repo and save it to diff --git a/src/config.lisp b/src/config.lisp index ad2304e..2802b5e 100644 --- a/src/config.lisp +++ b/src/config.lisp @@ -9,7 +9,7 @@ (plugins :initarg :plugins :accessor plugins) (repo :initarg :repo :accessor repo) (sitenav :initarg :sitenav :accessor sitenav) - (staging-dir :initarg :staging-dir :accessor staging-dir) + (staging-dir :initarg :staging-dir :accessor staging-dir :initform "/tmp/coleslaw/") (posts-dir :initarg :posts-dir :accessor posts-dir :initform "posts") (separator :initarg :separator :accessor separator :initform ";;;;;") (page-ext :initarg :page-ext :accessor page-ext :initform "html") @@ -40,7 +40,7 @@ are in the plugins folder in coleslaw's source directory." (destructuring-bind (name &rest args) plugin (apply 'enable-plugin (plugin-path name) args))))) -(defun discover-config-path (&optional (path "")) +(defun discover-config-path (path) "Check the supplied PATH for a .coleslawrc and if one doesn't exist, use the .coleslawrc in the home directory." (let ((custom-path (rel-path path ".coleslawrc"))) @@ -48,7 +48,7 @@ doesn't exist, use the .coleslawrc in the home directory." custom-path (rel-path (user-homedir-pathname) ".coleslawrc")))) -(defun load-config (config-key) +(defun load-config (&optional (config-key "")) "Load the coleslaw configuration from DIR/.coleslawrc, using CONFIG-KEY if necessary. DIR is ~ by default." (with-open-file (in (discover-config-path config-key))