Merge pull request #146 from guicho271828/roswell

roswell script

.travis.yml

@@ -20,3 +20,4 @@ cache:
 script:
   - ros -s prove -e "(ql:quickload '(coleslaw coleslaw-test))"
                  -e '(or (prove:run :coleslaw-test) (uiop:quit -1))'
+  - cli-tests/basic.sh

README.md

@@ -18,6 +18,9 @@ Have questions?
 
 See the [wiki](https://github.com/redline6561/coleslaw/wiki/Example-sites) for a list of coleslaw-powered blogs.
 
+Coleslaw should run on any conforming Common Lisp implementations but
+testing is primarily done on [SBCL](http://www.sbcl.org/) and [CCL](http://ccl.clozure.com/).
+
 ## Features
 
 * Git for storage
@@ -33,51 +36,84 @@ See the [wiki](https://github.com/redline6561/coleslaw/wiki/Example-sites) for a
 | [Pygments](http://pygments.org/)                       | [colorize](http://www.cliki.net/colorize)    |                                                       |
 
 
-## Installation
+## Installation/Tutorial
 
-Coleslaw should run on any conforming Common Lisp implementation but
+<!-- Don't let the first user select from multiple choises -->
-testing is primarily done on [SBCL](http://www.sbcl.org/) and
-[CCL](http://ccl.clozure.com/).
 
-Coleslaw can either be run **manually** on a local machine or
+Step 1: Install this library.
-triggered **automatically** on git push to a server.  If you want a
-server install, run these commands on your server _after_ setting up a
-[git bare repo](http://git-scm.com/book/en/Git-on-the-Server-Setting-Up-the-Server).
-Otherwise, run the commands on your local machine.
 
-1. Install a Common Lisp implementation (we recommend SBCL) and
+With [Roswell](https://roswell.github.io/),
-   [Quicklisp](http://quicklisp.org/).
+``` sh
-2. Place a config file for coleslaw in your `$HOME` directory. If you
+$ ros install coleslaw
-   want to run multiple blogs with coleslaw, you can keep each blog's
+$ export PATH="$HOME/.roswell/bin:$PATH" # If you haven't done this before
-   config file in that blog's repo.  Feel free to copy and edit the
+```
-   [example config][ex_config] or consult the [config docs][conf_docs]
-   to create one from scratch.
-3. This step depends on whether you're setting up a local or server install.
-   * Server Install: Copy and `chmod +x` the
-     [example post-receive hook][post_hook] to your blog's bare repo.
-   * Local Install:  Just run the following commands in the
-     REPL whenever you're ready to regenerate your blog:
-     ```
-     (ql:quickload :coleslaw)
-     (coleslaw:main "/path/to/my/blog/")
-     ```
-4. Optionally, point the web server of your liking at your config-specified
-   `:deploy-dir`. Or "deploy-dir/.curr" if the `versioned` plugin is enabled.
-5. If you use Emacs, consider installing
-   [coleslaw-mode](https://github.com/equwal/coleslaw-mode) to author your
-   posts.
 
-Now just write posts, git commit and build by hand or by push.
+or
 
-[ex_config]: https://github.com/redline6561/coleslaw/blob/master/examples/example.coleslawrc
+``` lisp
-[conf_docs]: https://github.com/redline6561/coleslaw/blob/master/docs/config.md
+(ql:quickload :coleslaw-cli)
-[post_hook]: https://github.com/redline6561/coleslaw/blob/master/examples/example.post-receive
+```
+
+
+Step 2: Initialize your blog repository.
+
+``` sh
+$ mkdir yourblog ; cd yourblog
+$ git init
+$ coleslaw setup
+```
+``` lisp
+(coleslaw-cli:setup)
+```
+
+Step 3: Write a post file in the current directory.
+The file should contain a certain metadata, so use the `coleslaw new` command,
+which instantiates a correct file for you.
+
+``` sh
+$ coleslaw new
+Created a post 2017-11-06.post .
+```
+``` lisp
+(coleslaw-cli:new "post")
+```
+
+Step 4: Generate the site from those post files.
+The result goes to the `deploy/` subdirectory.
+
+``` sh
+$ coleslaw
+```
+``` lisp
+(coleslaw-cli:generate)
+```
+
+Step 5: You can also launch a server...
+
+``` sh
+$ coleslaw preview
+```
+``` lisp
+(coleslaw-cli:preview)
+```
+
+Step 6: and watch the file system to automatically regenerate the site!
+
+``` sh
+$ coleslaw watch          # or even better,
+$ coleslaw watch-preview 
+```
+``` lisp
+(coleslaw-cli:watch)      ;; watch-preview does not work on REPL right now
+```
+
+For further customization, edit the `.coleslawrc` file generated by `coleslaw setup`.
+Consult the [config docs](https://github.com/redline6561/coleslaw/blob/master/docs/config.md).
 
 ## The Content Format
 
 Coleslaw expects content to have a file extension matching the class
-of the content. (I.e. `.post` for blog posts, `.page` for static
+of the content. (I.e. `.post` for blog posts, `.page` for static pages, etc.)
-pages, etc.)
 
 There should also be a metadata header on all files
 starting and ending with the config-specified `:separator`, ";;;;;" by
@@ -108,6 +144,20 @@ Two themes are provided: hyde, the default, and readable (based on
 A guide to creating themes for coleslaw lives
 [here](https://github.com/redline6561/coleslaw/blob/master/docs/themes.md).
 
+## Deploying on a standalone server
+
+Coleslaw can deploy to a standalone server.
+If you want this server installation, initialize a bare git repo and
+set up the post-receive hook on that repo.
+
+* First initialize a [git bare repo](http://git-scm.com/book/en/Git-on-the-Server-Setting-Up-the-Server) on the server.
+* Copy [example post-receive hook][post_hook] to your blog's bare repo and set the executable bit (`chmod +x`).
+
+* Point the web server at `:deploy-dir` attribute on the config file.
+  Or "deploy-dir/.curr" if the `versioned` plugin is enabled.
+
+[post_hook]: https://github.com/redline6561/coleslaw/blob/master/examples/example.post-receive
+
 ## Hacking
 
 A core goal of *coleslaw* is to be both pleasant to read and easy to

cli-tests/basic.sh

@@ -0,0 +1,39 @@
+#!/bin/bash -x
+
+set -e
+
+dir=$(mktemp -d)
+
+cd $dir
+
+coleslaw setup
+
+cat .coleslawrc
+
+post=$(coleslaw new post "my first blog")
+
+echo "my firrrrrrst text!!!!" >> "$post"
+
+cat "$post"
+
+coleslaw generate
+
+coleslaw preview &
+pid=$!
+
+trap "kill $pid; rm -rf $dir" EXIT
+
+sleep 3
+
+wget 127.0.0.1:5000 -O-
+
+! wget 127.0.0.1:5000/nosuchurl -O-
+
+# (
+#     wget 127.0.0.1:5000/nosuchurl -O-
+#     echo $?
+#     true
+# )
+
+
+

cli/cli.lisp

@@ -0,0 +1,292 @@
+(defpackage :coleslaw-cli
+  (:use :cl :trivia)
+  (:export
+   #:copy-theme
+   #:setup
+   #:new
+   #:generate
+   #:preview
+   #:watch
+   #:watch-preview
+   #:help))
+  
+(in-package :coleslaw-cli)
+
+(defun setup-coleslawrc (user &aux (path (merge-pathnames ".coleslawrc")))
+  "Set up the default .coleslawrc file in the current directory."
+  (with-open-file (s path :direction :output :if-exists :supersede :if-does-not-exist :create)
+    (format t "~&Generating ~a ...~%" path)
+    ;; odd formatting in this source code because emacs has problem detecting the parenthesis inside a string
+    (format s ";;; -*- mode : lisp -*-~%(~
+ ;; Required information
+ :author \"~a\"                         ;; to be placed on post pages and in the copyright/CC-BY-SA notice
+ :deploy-dir \"deploy/\"                ;; for Coleslaw's generated HTML to go in
+ :domain \"\"                           ;; to generate absolute links to the site content
+ :routing ((:post           \"posts/~~a\") ;; to determine the URL scheme of content on the site
+           (:tag-index      \"tag/~~a\")
+           (:month-index    \"date/~~a\")
+           (:numeric-index  \"~~d\")
+           (:feed           \"~~a.xml\")
+           (:tag-feed       \"tag/~~a.xml\"))
+ :title \"Improved Means for Achieving Deteriorated Ends\" ;; a site title
+ :theme \"hyde\"                        ;; to select one of the themes in \"coleslaw/themes/\"
+ 
+ ;; Optional information
+ :excerpt-sep \"<!--more-->\"           ;; to set the separator for excerpt in content 
+ :feeds (\"lisp\")
+ :plugins ((analytics :tracking-code \"foo\")
+           (disqus :shortname \"my-site-name\")
+           ; (incremental)  ;; *Remove comment to enable incremental builds.
+           (mathjax)
+           (sitemap)
+           (static-pages)
+           ; (versioned)    ;; *Remove comment to enable symlinked, timestamped deploys.
+          )
+ :sitenav ((:url \"http://~a.github.com/\" :name \"Home\")
+           (:url \"http://twitter.com/~a\" :name \"Twitter\")
+           (:url \"http://github.com/~a\" :name \"Code\")
+           (:url \"http://soundcloud.com/~a\" :name \"Music\")
+           (:url \"http://redlinernotes.com/docs/talks/\" :name \"Talks\"))
+ :staging-dir \"/tmp/coleslaw/\"  ;; for Coleslaw to do intermediate work, default: \"/tmp/coleslaw\"
+)
+
+;; * Prerequisites described in plugin docs."
+            user
+            user
+            user
+            user
+            user)))
+
+(defun copy-theme (which &optional (target which))
+  "Copy the theme named WHICH into the blog directory and rename it into TARGET"
+  (format t "~&Copying themes/~a ...~%" which)
+  (if (probe-file (format nil "themes/~a" which))
+      (format t "~&  themes/~a already exists.~%" which)
+      (progn
+        (ensure-directories-exist "themes/" :verbose t)
+        (uiop:run-program `("cp" "-v" "-r"
+                                 ,(namestring (coleslaw::app-path "themes/~a/" which))
+                                 ,(namestring (merge-pathnames (format nil "themes/~a" target))))))))
+
+(defun setup (&optional (user (uiop:getenv "USER")))
+  (setup-coleslawrc user)
+  (copy-theme "hyde" "default"))
+
+(defun read-rc (&aux (path (merge-pathnames ".coleslawrc")))
+  (with-open-file (s (if (probe-file path)
+                         path
+                         (merge-pathnames #p".coleslawrc" (user-homedir-pathname))))
+    (read s)))
+
+(defun new (&optional (type "post") name)
+  (let ((sep (getf (read-rc) :separator ";;;;;")))
+    (multiple-value-match (get-decoded-time)
+      ((second minute hour date month year _ _ _)
+       (let* ((name (or name
+                        (format nil "~a-~2,,,'0@a-~2,,,'0@a" year month date)))
+              (path (merge-pathnames (make-pathname :name name :type type))))
+         (with-open-file (s path
+                            :direction :output :if-exists :error :if-does-not-exist :create)
+           (format s "~
+~a
+title: ~a
+tags: bar, baz
+date: ~a-~2,,,'0@a-~2,,,'0@a ~2,,,'0@a:~2,,,'0@a:~2,,,'0@a
+format: md
+~:[~*~;URL: pages/~a.html~%~]~
+~a
+
+<!-- **** your post here (remove this line) **** -->
+<!-- format: could be 'html' (for raw html) or 'md' (for markdown).  -->
+
+Here is my content.
+
+<!--more-->
+
+Excerpt separator can also be extracted from content.
+Add `excerpt: <string>` to the above metadata.
+Excerpt separator is `<!--more-->` by default.
+"
+                   sep
+                   name
+                   year month date hour minute second
+                   (string= type "page") name
+                   sep)
+           (format *error-output* "~&Created a ~a \"~a\".~%" type name)
+           (format t "~&~a~%" path)
+           path))))))
+
+(defun generate ()
+  (coleslaw:main *default-pathname-defaults*))
+
+(defun preview (&optional (path (getf (read-rc) :deploy-dir)))
+  ;; clack depends on the global binding of *default-pathname-defaults*.
+  (let ((oldpath *default-pathname-defaults*))
+    (unwind-protect
+         (progn
+           (when path
+             (setf *default-pathname-defaults* (truename path)))
+           (format t "~%Starting a Clack server at ~a~%" path)
+           (clack:clackup
+            (lack:builder
+             :accesslog
+             (:static :path (lambda (p)
+                              (if (char= #\/ (alexandria:last-elt p))
+                                  (concatenate 'string p "index.html")
+                                  p)))
+             #'identity)
+            :use-thread nil))
+      (setf *default-pathname-defaults* oldpath))))
+
+;; code from fs-watcher
+
+(defun mtime (pathname)
+  "Returns the mtime of a pathname"
+  (when (ignore-errors (probe-file pathname))
+    (file-write-date pathname)))
+
+(defun dir-contents (pathnames test)
+  (remove-if-not test
+                 ;; uiop:slurp-input-stream
+                 (uiop:run-program `("find" ,@(mapcar #'namestring pathnames))
+                                   :output :lines)))
+
+(defun run-loop (pathnames mtimes callback delay)
+  "The main loop constantly polling the filesystem"
+  (loop
+    (sleep delay)
+    (map nil
+         #'(lambda (pathname)
+             (let ((mtime (mtime pathname)))
+               (unless (eql mtime (gethash pathname mtimes))
+                 (funcall callback pathname)
+                 (if mtime
+                   (setf (gethash pathname mtimes) mtime)
+                   (remhash pathname mtimes)))))
+         pathnames)))
+
+(defun watch (&optional (source-path *default-pathname-defaults*))
+  (format t "~&Start watching! : ~a~%" source-path)
+  (let ((pathnames 
+         (dir-contents (list source-path)
+                       (lambda (p) (not (equal "fasl" (pathname-type p))))))
+        (mtimes (make-hash-table)))
+    (dolist (pathname pathnames)
+      (setf (gethash pathname mtimes) (mtime pathname)))
+    (ignore-errors
+      (run-loop pathnames
+                mtimes
+                (lambda (pathname)
+                  (format t "~&Changes detected! : ~a~%" pathname)
+                  (finish-output)
+                  (handler-case
+                      (coleslaw:main source-path)
+                    (error (c)
+                      (format *error-output* "something happened... ~a" c))))
+                1))))
+
+(defun watch-preview (&optional (source-path *default-pathname-defaults*))
+  (when (member :swank *features*)
+    (warn "FIXME: This command does not do what you intend from a SLIME session."))
+  (ignore-errors
+    (uiop:run-program
+     ;; The hackiness here is because clack fails? to handle? SIGINT correctly when run in a threaded mode
+     `("sh" "-c" ,(format nil "coleslaw watch ~a &~
+                               coleslaw preview &~
+                               jobs -p;~
+                               trap \"kill $(jobs -p)\" EXIT;~
+                               wait" source-path))
+     :output :interactive
+     :error-output :interactive)))
+
+(defun help ()
+  (format *error-output* "
+
+
+Coleslaw, a Flexible Lisp Blogware.
+Written by: Brit Butler <redline6561@gmail.com>.
+Distributed by BSD license.
+
+Command Line Syntax:
+
+coleslaw setup [NAME]                --- Sets up a new .coleslawrc file in the current directory.
+coleslaw copy-theme THEME [TARGET]   --- Copies the installed THEME in coleslaw to the current directory with a different name TARGET.
+coleslaw new [TYPE] [NAME]           --- Creates a new content file with the correct format. TYPE defaults to 'post', NAME defaults to the current date.
+coleslaw generate                    --- Generates the static html according to .coleslawrc .
+coleslaw preview [DIRECTORY]         --- Runs a preview server at port 5000. DIRECTORY defaults to the deploy directory (described in .coleslawrc).
+coleslaw watch [DIRECTORY]           --- Watches the given directory and generates the site when changes are detected. Defaults to the current directory.
+coleslaw                             --- Shorthand of 'coleslaw generate'.
+coleslaw -h                          --- Show this help
+
+Corresponding REPL commands are available in coleslaw-cli package.
+
+```lisp
+  (ql:quickload :coleslaw-cli)
+  (coleslaw-cli:setup            &optional name)
+  (coleslaw-cli:copy-theme theme &optional target)
+  (coleslaw-cli:new              &optional type name)
+  (coleslaw-cli:generate)
+  (coleslaw-cli:preview &optional directory)
+  (coleslaw-cli:watch   &optional directory)
+```
+
+Examples:
+
+* set up a blog
+
+    mkdir yourblog ; cd yourblog
+    git init
+    coleslaw setup
+    git commit -a -m 'initial repo'
+
+* Copy the base theme to the current directory for modification
+
+    coleslaw copy-theme hyde mytheme
+
+* Create a post
+
+    coleslaw new
+
+* Create a page (static page)
+
+    coleslaw new page
+
+* Generate a site
+
+    coleslaw generate
+    # or just:
+    coleslaw
+
+* Preview a site
+
+    coleslaw preview
+    # or
+    coleslaw preview .
+
+"
+          ))
+
+(defun main (&rest argv)
+  (declare (ignorable argv))
+  (match argv
+    ((list* "setup" rest)
+     (apply #'setup rest))
+    ((list* "preview" rest)
+     (apply #'preview rest))
+    ((list* "watch" rest)
+     (apply #'watch rest))
+    ((list* "watch-preview" rest)
+     (apply #'watch-preview rest))
+    ((list* "new" rest)
+     (apply #'new rest))
+    ((or nil (list "generate"))
+     (generate))
+    ((list* "copy-theme" rest)
+     (apply #'copy-theme rest))
+    ((list* (or "-v" "--version") _)
+     )
+    ((list* (or "-h" "--help") _)
+     (help))))
+
+(when (member :swank *features*)
+  (help))

coleslaw-cli.asd

@@ -0,0 +1,14 @@
+(defsystem #:coleslaw-cli
+  :name "coleslaw"
+  :description "Flexible Lisp Blogware"
+  :version "0.9.7"
+  :license "BSD"
+  :author "Brit Butler <redline6561@gmail.com>"
+  :pathname "cli/"
+  :depends-on (:coleslaw
+               :clack
+               :trivia
+               :uiop)
+  :serial t
+  :components ((:file "cli")))
+

coleslaw-test.asd

@@ -4,10 +4,11 @@
   :description "A test suite for coleslaw."
   :license "BSD"
   :author "Brit Butler <redline6561@gmail.com>"
-  :depends-on (:coleslaw :prove)
+  :depends-on (:coleslaw :coleslaw-cli :prove)
   :defsystem-depends-on (:prove-asdf)
   :components ((:module "tests"
                 :components
-                        ((:test-file "tests"))))
+                ((:test-file "tests")
+                 (:test-file "cli"))))
   :perform (test-op :after (op c)
                     (uiop:symbol-call :prove 'run c)))

roswell/coleslaw.ros

@@ -0,0 +1,10 @@
+#!/bin/sh
+#|-*- mode:lisp -*-|#
+#|
+exec ros -Q -L sbcl-bin -m coleslaw -- $0 "$@"
+|#
+(progn ;;init forms
+  (ros:ensure-asdf)
+  #+quicklisp (ql:quickload '(:coleslaw-cli) :silent t))
+
+(in-package :coleslaw-cli)

src/coleslaw.lisp

@@ -43,7 +43,7 @@ in REPO-DIR. Optionally, OLDREV is the revision prior to the last push."
 (defgeneric deploy (staging)
   (:documentation "Deploy the STAGING build to the directory specified in the config.")
   (:method (staging)
-    (run-program "rsync --delete -avz ~a ~a" staging (deploy-dir *config*))))
+    (run-program "rsync --delete -avz ~a ~a" staging (merge-pathnames (deploy-dir *config*)))))
 
 (defun update-symlink (path target)
   "Update the symlink at PATH to point to TARGET."

tests/cli.lisp

@@ -0,0 +1,23 @@
+
+(in-package :coleslaw-tests)
+
+(plan 2)
+
+(let ((*default-pathname-defaults*
+       (pathname
+        (format nil "~a/"
+                (uiop:run-program `("mktemp" "-d")
+                                  :output `(:string :stripped t))))))
+  (coleslaw-cli:setup)
+  (let ((file (coleslaw-cli:new)))
+    (ok (probe-file file)))
+  (coleslaw-cli:generate)
+  (print (format nil "~adeploy/index.html" *default-pathname-defaults*))
+  (ok (probe-file (format nil "~adeploy/index.html" *default-pathname-defaults*))))
+
+(finalize)
+
+  
+  
+
+