Update HACKING and TODO

HACKING.rst

@@ -48,8 +48,8 @@ An artifact is an abstract representation of something that will produce output
 when it is written. It can be something like a file that will be copied
 verbatim, or a potential HTML page. Artifact can also be of higher order,
 meaning they wrap around other artifacts. The compound artifact is a simple
-case of just a plain container artifact. On the other hand, the blog artifact
-also has its own data such as the blog title.
+case of just a plain container artifact. On the other hand, the blog is a
+higher-order artifact that also has its own data such as the blog title.
 
 Metadata
 ========
@@ -87,6 +87,81 @@ Usually readers will be stored in a key-value variable where the key is the
 file format. Another function can then dispatch on the file format to the
 correct reader.
 
+File systems and instructions
+=============================
+
+Ultimately every artifact will be written to a file. Hard-coding this would
+require intermixing the concern of "represents some content" and "will produce
+a file". The lack of separation of concerns makes the artifact classes harder
+to reason about and harder to test. The act of producing an actual on-disk file
+needs to be separate from the individual artifacts. The usual solution would be
+to have some sort of "file system service" object which we can inject it as a
+dependency into a function, but this is not an elegant solution.
+
+My solution are file systems and instructions. A file system is an object which
+abstracts away access to an actual file system. An instruction is an object
+which represents an action to perform, such as copying a file or writing a
+string to a file.
+
+.. code:: lisp
+
+   (defclass base-file-system (file-system)
+     ((directory :initarg :directory :reader file-system-directory :type pathname
+        :documentation "Actual directory within the file system."))
+     (:documentation
+      "A file system which accesses files of the host OS relative to a given base
+   directory."))
+
+   (defclass write-string-contents (file-system-instruction)
+     ((path :initarg :path :reader instruction-path :type pathname
+         :documentation "Path to the output file")
+      (contents :initarg :contents :initform (make-string 0) :type string
+        :documentation "The file content as a string"))
+     (:documentation
+      "Instruction which creates a file with the given content."))
+
+We can apply an instruction to a file system, which carries out the action in
+the directory of the file system. We use the fact that CLOS supports multiple
+dispatch to dispatch on both the file system and the instruction.
+
+.. code:: lisp
+
+   (defmethod write-to-filesystem ((instruction write-string-contents)
+                                   (file-system base-file-system))
+     "A primitive implementation producing one file for fixed contents and an
+     absolute file system."
+     (let ((path (fad:merge-pathnames-as-file
+                   (fad:pathname-as-directory (file-system-directory file-system))
+                   (instruction-path instruction))))
+       (with-slots (contents) instruction
+         (write-string-to-file contents path))))
+
+But wait, if we have `n` file systems and `m` instructions, does that mean we
+need `n * m` implementations? No, most file systems and instructions are
+actually of a higher-order and reduce down to the most elemental ones. Consider
+the compound instruction, a container instruction which wraps other
+instructions:
+
+.. code:: lisp
+
+   (defmethod write-to-filesystem ((instruction compound-instruction)
+                                   file-system)
+     (with-slots (instructions) instruction
+       (dolist (instruction instructions)
+         (write-to-filesystem instruction file-system))))
+
+Instructions a produced by deriving an artifact; e.g. to derive an HTML
+artifact we generate the file contents as a string and produce a
+`WRITE-STRING-CONTENTS` instruction with the content and file name. We do not
+care where the file is written to, that part is the responsibility of the file
+system.
+
+The most elemental file system simply references an on-disc directory. A
+higher-order file system is the overlay file system which adds a path on top of
+another file system. We are not bound by on-disc directories though: an FTP
+file system could abstract away access to an FTP server, a ZIP file system
+might abstract away access to a ZIP file.
+
 
 The blog plugin
 ###############

TODO.rst

@@ -8,36 +8,6 @@
 Core
 ####
 
-New feature: file systems
-=========================
-
-Currently file output is strongly coupled to the file system of the OS. If we
-want to write an artifact, then writing the artifact is the responsibility of
-the artifact: it performs the low-level file system access, it generates the
-output text and it manages the file names, including the output directory path.
-
-My proposal is to add a lever of indirection by separating concerns. There are
-three participants:
-
-- File systems
-- Artifacts
-- Instructions
-
-The artifact is an abstract representation of one or more future files. It is
-then *derived* to produce a low-level instruction on what action to actually
-perform to produce the file (relative file name, contents). The file system
-interprets the instruction by accessing the file systems and outputting the
-actual contents.
-
-All this will be implemented using CLOS. A generic function dispatches on both
-instruction and file system. There will be core implementations for elemental
-instructions and file systems. Implementations for new classes will then be
-implemented on top of these primitive methods. For example, an implementation
-for an instruction which produces multiple files will created multiple
-lower-level instructions and call the generic function for each of these
-instructions and the original file system.
-
-
 New feature: sources
 ====================
 
@@ -71,9 +41,6 @@ Cleanup
 Testing
 =======
 
-- Artifacts
-  - Come up with a proper artifacts interface
-  - Test the individual artifact implementations (function `WRITE-ARTIFACT`)
 - Update reader tests to public interface once it is done
 
 

src/hssg/filesystem.lisp

@@ -61,7 +61,9 @@ directory."))
 ;;; --- INSTRUCTION CLASSES ---------------------------------------------------
 (defclass write-string-contents (path-instruction)
   ((contents :initarg :contents :initform (make-string 0) :type string
-     :documentation "The file content as a string")))
+     :documentation "The file content as a string"))
+  (:documentation
+   "Instruction which creates a file with the given content."))
 
 (defclass copy-file (path-instruction)
   ((base-path :initarg :base-path :type pathname