marcuskammer/emacs.d
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
emacs.d/clones/lispcookbook.github.io/cl-cookbook/databases.html

953 lines
33 KiB
HTML
Raw Normal View History

Add clones websites
2022-08-02 12:34:59 +02:00
<!DOCTYPE html>
<html lang="en">
<head>
<meta name="generator" content=
"HTML Tidy for HTML5 for Linux version 5.2.0">
<title>Database Access and Persistence</title>
<meta charset="utf-8">
<meta name="description" content="A collection of examples of using Common Lisp">
<meta name="viewport" content=
"width=device-width, initial-scale=1">
<link rel="stylesheet" href=
"assets/style.css">
<script type="text/javascript" src=
"assets/highlight-lisp.js">
</script>
<script type="text/javascript" src=
"assets/jquery-3.2.1.min.js">
</script>
<script type="text/javascript" src=
"assets/jquery.toc/jquery.toc.min.js">
</script>
<script type="text/javascript" src=
"assets/toggle-toc.js">
</script>
<link rel="stylesheet" href=
"assets/github.css">
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
</head>
<body>
<h1 id="title-xs"><a href="index.html">The Common Lisp Cookbook</a> &ndash; Database Access and Persistence</h1>
<div id="logo-container">
<a href="index.html">
<img id="logo" src="assets/cl-logo-blue.png"/>
</a>
<div id="searchform-container">
<form onsubmit="duckSearch()" action="javascript:void(0)">
<input id="searchField" type="text" value="" placeholder="Search...">
</form>
</div>
<div id="toc-container" class="toc-close">
<div id="toc-title">Table of Contents</div>
<ul id="toc" class="list-unstyled"></ul>
</div>
</div>
<div id="content-container">
<h1 id="title-non-xs"><a href="index.html">The Common Lisp Cookbook</a> &ndash; Database Access and Persistence</h1>
<!-- Announcement we can keep for 1 month or more. I remove it and re-add it from time to time. -->
<p class="announce">
📹 <a href="https://www.udemy.com/course/common-lisp-programming/?couponCode=6926D599AA-LISP4ALL">NEW! Learn Lisp in videos and support our contributors with this 40% discount.</a>
</p>
<p class="announce-neutral">
📕 <a href="index.html#download-in-epub">Get the EPUB and PDF</a>
</p>
<div id="content"
<p>The
<a href="https://github.com/CodyReichert/awesome-cl#database">Database section on the Awesome-cl list</a>
is a resource listing popular libraries to work with different kind of
databases. We can group them roughly in four categories:</p>
<ul>
<li>wrappers to one database engine (cl-sqlite, postmodern, cl-redis,…),</li>
<li>interfaces to several DB engines (clsql, sxql,…),</li>
<li>persistent object databases (bknr.datastore (see chap. 21 of “Common Lisp Recipes”), ubiquitous,…),</li>
<li><a href="https://en.wikipedia.org/wiki/Object-relational_mapping">Object Relational Mappers</a> (Mito),</li>
</ul>
<p>and other DB-related tools (pgloader).</p>
<p>We’ll begin with an overview of Mito. If you must work with an
existing DB, you might want to have a look at cl-dbi and clsql. If you
don’t need a SQL database and want automatic persistence of Lisp
objects, you also have a choice of libraries.</p>
<h2 id="the-mito-orm-and-sxql">The Mito ORM and SxQL</h2>
<p>Mito is in Quicklisp:</p>
<pre><code class="language-lisp">(ql:quickload "mito")
</code></pre>
<h3 id="overview">Overview</h3>
<p><a href="https://github.com/fukamachi/mito">Mito</a> is “an ORM for Common Lisp
with migrations, relationships and PostgreSQL support”.</p>
<ul>
<li>it <strong>supports MySQL, PostgreSQL and SQLite3</strong>,</li>
<li>when defining a model, it adds an <code>id</code> (serial primary key),
<code>created_at</code> and <code>updated_at</code> fields by default like Ruby’s
ActiveRecord or Django,</li>
<li>handles DB <strong>migrations</strong> for the supported backends,</li>
<li>permits DB <strong>schema versioning</strong>,</li>
<li>is tested under SBCL and CCL.</li>
</ul>
<p>As an ORM, it allows to write class definitions, to specify relationships, and
provides functions to query the database. For custom queries, it relies on
<a href="https://github.com/fukamachi/sxql">SxQL</a>, an SQL generator that provides the
same interface for several backends.</p>
<p>Working with Mito generally involves these steps:</p>
<ul>
<li>connecting to the DB</li>
<li>writing <a href="clos.html">CLOS</a> classes to define models</li>
<li>running migrations to create or alter tables</li>
<li>creating objects, saving same in the DB,</li>
</ul>
<p>and iterating.</p>
<h3 id="connecting-to-a-db">Connecting to a DB</h3>
<p>Mito provides the function <code>connect-toplevel</code> to establish a
connection to RDBMs:</p>
<pre><code class="language-lisp">(mito:connect-toplevel :mysql :database-name "myapp" :username "fukamachi" :password "c0mon-1isp")
</code></pre>
<p>The driver type can be of <code>:mysql</code>, <code>:sqlite3</code> and <code>:postgres</code>.</p>
<p>With sqlite you don’t need the username and password:</p>
<pre><code class="language-lisp">(mito:connect-toplevel :sqlite3 :database-name "myapp")
</code></pre>
<p>As usual, you need to create the MySQL or PostgreSQL database beforehand.
Refer to their documentation.</p>
<p>Connecting sets <code>mito:*connection*</code> to the new connection and returns it.</p>
<p>Disconnect with <code>disconnect-toplevel</code>.</p>
<p>You might make good use of a wrapper function:</p>
<pre><code class="language-lisp">(defun connect ()
"Connect to the DB."
(mito:connect-toplevel :sqlite3 :database-name "myapp"))
</code></pre>
<h3 id="models">Models</h3>
<h4 id="defining-models">Defining models</h4>
<p>In Mito, you can define a class which corresponds to a database table with the <code>deftable</code> macro:</p>
<pre><code class="language-lisp">(mito:deftable user ()
((name :col-type (:varchar 64))
(email :col-type (or (:varchar 128) :null)))
</code></pre>
<p>Alternatively, you can specify <code>(:metaclass mito:dao-table-class)</code> in a regular class definition.</p>
<p>The <code>deftable</code> macro automatically adds some slots: a primary key named <code>id</code> if there’s no primary key, and <code>created_at</code> and <code>updated_at</code> for recording timestamps. Specifying <code>(:auto-pk nil)</code> and <code>(:record-timestamps nil)</code> in the <code>deftable</code> form will disable these behaviours. A <code>deftable</code> class will also come with initializers, named after the slot, and accessors, of form <code>&lt;class-name&gt;-&lt;slot-name&gt;</code>, for each named slot. For example, for the <code>name</code> slot in the above table definition, the initarg <code>:name</code> will be added to the constuctor, and the accessor <code>user-name</code> will be created.</p>
<p>You can inspect the new class:</p>
<pre><code class="language-lisp">(mito.class:table-column-slots (find-class 'user))
;=&gt; (#&lt;MITO.DAO.COLUMN:DAO-TABLE-COLUMN-CLASS MITO.DAO.MIXIN::ID&gt;
; #&lt;MITO.DAO.COLUMN:DAO-TABLE-COLUMN-CLASS COMMON-LISP-USER::NAME&gt;
; #&lt;MITO.DAO.COLUMN:DAO-TABLE-COLUMN-CLASS COMMON-LISP-USER::EMAIL&gt;
; #&lt;MITO.DAO.COLUMN:DAO-TABLE-COLUMN-CLASS MITO.DAO.MIXIN::CREATED-AT&gt;
; #&lt;MITO.DAO.COLUMN:DAO-TABLE-COLUMN-CLASS MITO.DAO.MIXIN::UPDATED-AT&gt;)
</code></pre>
<p>The class inherits <code>mito:dao-class</code> implicitly.</p>
<pre><code class="language-lisp">(find-class 'user)
;=&gt; #&lt;MITO.DAO.TABLE:DAO-TABLE-CLASS COMMON-LISP-USER::USER&gt;
(c2mop:class-direct-superclasses *)
;=&gt; (#&lt;STANDARD-CLASS MITO.DAO.TABLE:DAO-CLASS&gt;)
</code></pre>
<p>This may be useful when you define methods which can be applied for
all table classes.</p>
<p>For more information on using the Common Lisp Object System, see the
<a href="clos.html">clos</a> page.</p>
<h4 id="creating-the-tables">Creating the tables</h4>
<p>After defining the models, you must create the tables:</p>
<pre><code class="language-lisp">(mito:ensure-table-exists 'user)
</code></pre>
<p>So a helper function:</p>
<pre><code class="language-lisp">(defun ensure-tables ()
(mapcar #'mito:ensure-table-exists '(user foo bar)))
</code></pre>
<p>See
<a href="https://github.com/fukamachi/mito#generating-table-definitions">Mito’s documentation</a>
for a couple more ways.</p>
<p>When you alter the model you’ll need to run a DB migration, see the next section.</p>
<h4 id="fields">Fields</h4>
<h5 id="fields-types">Fields types</h5>
<p>Field types are:</p>
<p><code>(:varchar &lt;integer&gt;)</code> ,</p>
<p><code>:serial</code>, <code>:bigserial</code>, <code>:integer</code>, <code>:bigint</code>, <code>:unsigned</code>,</p>
<p><code>:timestamp</code>, <code>:timestamptz</code>,</p>
<p><code>:bytea</code>,</p>
<h5 id="optional-fields">Optional fields</h5>
<p>Use <code>(or &lt;real type&gt; :null)</code>:</p>
<pre><code class="language-lisp"> (email :col-type (or (:varchar 128) :null))
</code></pre>
<h5 id="field-constraints">Field constraints</h5>
<p><code>:unique-keys</code> can be used like so:</p>
<pre><code class="language-lisp">(mito:deftable user ()
((name :col-type (:varchar 64))
(email :col-type (:varchar 128))
(:unique-keys email))
</code></pre>
<p>We already saw <code>:primary-key</code>.</p>
<p>You can change the table name with <code>:table-name</code>.</p>
<h4 id="relationships">Relationships</h4>
<p>You can define a relationship by specifying a foreign class with <code>:col-type</code>:</p>
<pre><code class="language-lisp">(mito:deftable tweet ()
((status :col-type :text)
;; This slot refers to USER class
(user :col-type user))
(table-definition (find-class 'tweet))
;=&gt; (#&lt;SXQL-STATEMENT: CREATE TABLE tweet (
; id BIGSERIAL NOT NULL PRIMARY KEY,
; status TEXT NOT NULL,
; user_id BIGINT NOT NULL,
; created_at TIMESTAMP,
; updated_at TIMESTAMP
; )&gt;)
</code></pre>
<p>Now you can create or retrieve a <code>TWEET</code> by a <code>USER</code> object, not a <code>USER-ID</code>.</p>
<pre><code class="language-lisp">(defvar *user* (mito:create-dao 'user :name "Eitaro Fukamachi"))
(mito:create-dao 'tweet :user *user*)
(mito:find-dao 'tweet :user *user*)
</code></pre>
<p>Mito doesn’t add foreign key constraints for referring tables.</p>
<h5 id="one-to-one">One-to-one</h5>
<p>A one-to-one relationship is simply represented with a simple foreign
key on a slot (as <code>:col-type user</code> in the <code>tweet</code> class). Besides, we
can add a unicity constraint, as with <code>(:unique-keys email)</code>.</p>
<h5 id="one-to-many-many-to-one">One-to-many, many-to-one</h5>
<p>The tweet example above shows a one-to-many relationship between a user and
his tweets: a user can write many tweets, and a tweet belongs to only
one user.</p>
<p>The relationship is defined with a foreign key on the “many” side
linking back to the “one” side. Here the <code>tweet</code> class defines a
<code>user</code> foreign key, so a tweet can only have one user. You didn’t need
to edit the <code>user</code> class.</p>
<p>A many-to-one relationship is actually the contrary of a one-to-many.
You have to put the foreign key on the appropriate side.</p>
<h5 id="many-to-many">Many-to-many</h5>
<p>A many-to-many relationship needs an intermediate table, which will be
the “many” side for the two tables it is the intermediary of.</p>
<p>And, thanks to the join table, we can store more information about the relationship.</p>
<p>Let’s define a <code>book</code> class:</p>
<pre><code class="language-lisp">(mito:deftable book ()
((title :col-type (:varchar 128))
(ean :col-type (or (:varchar 128) :null))))
</code></pre>
<p>A user can have many books, and a book (as the title, not the physical
copy) is likely to be in many people’s library. Here’s the
intermediate class:</p>
<pre><code class="language-lisp">(mito:deftable user-books ()
((user :col-type user)
(book :col-type book)))
</code></pre>
<p>Each time we want to add a book to a user’s collection (say in
a <code>add-book</code> function), we create a new <code>user-books</code> object.</p>
<p>But someone may very well own many copies of one book. This is an
information we can store in the join table:</p>
<pre><code class="language-lisp">(mito:deftable user-books ()
((user :col-type user)
(book :col-type book)
;; Set the quantity, 1 by default:
(quantity :col-type :integer)))
</code></pre>
<h4 id="inheritance-and-mixin">Inheritance and mixin</h4>
<p>A subclass of DAO-CLASS is allowed to be inherited. This may be useful
when you need classes which have similar columns:</p>
<pre><code class="language-lisp">(mito:deftable user ()
((name :col-type (:varchar 64))
(email :col-type (:varchar 128)))
(:unique-keys email))
(mito:deftable temporary-user (user)
((registered-at :col-type :timestamp)))
(mito:table-definition 'temporary-user)
;=&gt; (#&lt;SXQL-STATEMENT: CREATE TABLE temporary_user (
; id BIGSERIAL NOT NULL PRIMARY KEY,
; name VARCHAR(64) NOT NULL,
; email VARCHAR(128) NOT NULL,
; registered_at TIMESTAMP NOT NULL,
; created_at TIMESTAMP,
; updated_at TIMESTAMP,
; UNIQUE (email)
; )&gt;)
</code></pre>
<p>If you need a ‘template’ for tables which aren’t related to any
database tables, you can use <code>DAO-TABLE-MIXIN</code> in a <code>defclass</code> form. The <code>has-email</code>
class below will not create a table.</p>
<pre><code class="language-lisp">(defclass has-email ()
((email :col-type (:varchar 128)
:initarg :email
:accessor object-email))
(:metaclass mito:dao-table-mixin)
(:unique-keys email))
;=&gt; #&lt;MITO.DAO.MIXIN:DAO-TABLE-MIXIN COMMON-LISP-USER::HAS-EMAIL&gt;
(mito:deftable user (has-email)
((name :col-type (:varchar 64))))
;=&gt; #&lt;MITO.DAO.TABLE:DAO-TABLE-CLASS COMMON-LISP-USER::USER&gt;
(mito:table-definition 'user)
;=&gt; (#&lt;SXQL-STATEMENT: CREATE TABLE user (
; id BIGSERIAL NOT NULL PRIMARY KEY,
; name VARCHAR(64) NOT NULL,
; email VARCHAR(128) NOT NULL,
; created_at TIMESTAMP,
; updated_at TIMESTAMP,
; UNIQUE (email)
; )&gt;)
</code></pre>
<p>See more examples of use in <a href="https://github.com/fukamachi/mito-auth/">mito-auth</a>.</p>
<h4 id="troubleshooting">Troubleshooting</h4>
<h5 id="cannot-change-class-objects-into-class-metaobjects">“Cannot CHANGE-CLASS objects into CLASS metaobjects.”</h5>
<p>If you get the following error message:</p>
<pre><code>Cannot CHANGE-CLASS objects into CLASS metaobjects.
[Condition of type SB-PCL::METAOBJECT-INITIALIZATION-VIOLATION]
See also:
The Art of the Metaobject Protocol, CLASS [:initialization]
</code></pre>
<p>it is certainly because you first wrote a class definition and <em>then</em>
added the Mito metaclass and tried to evaluate the class definition
again.</p>
<p>If this happens, you must remove the class definition from the current package:</p>
<pre><code class="language-lisp">(setf (find-class 'foo) nil)
</code></pre>
<p>or, with the Slime inspector, click on the class and find the “remove” button.</p>
<p>More info <a href="https://stackoverflow.com/questions/38811931/how-to-change-classs-metaclass">here</a>.</p>
<h3 id="migrations">Migrations</h3>
<p>We can run database migrations manually, as shown below, or we can
automatically run migrations after a change to the model
definitions. To enable automatic migrations, set <code>mito:*auto-migration-mode*</code> to <code>t</code>.</p>
<p>The first step is to create the tables, if needed:</p>
<pre><code class="language-lisp">(ensure-table-exists 'user)
</code></pre>
<p>then alter the tables:</p>
<pre><code class="language-lisp">(mito:migrate-table 'user)
</code></pre>
<p>You can check the SQL generated code with <code>migration-expressions
'class</code>. For example, we create the <code>user</code> table:</p>
<pre><code class="language-lisp">(ensure-table-exists 'user)
;-&gt; ;; CREATE TABLE IF NOT EXISTS "user" (
; "id" BIGSERIAL NOT NULL PRIMARY KEY,
; "name" VARCHAR(64) NOT NULL,
; "email" VARCHAR(128),
; "created_at" TIMESTAMP,
; "updated_at" TIMESTAMP
; ) () [0 rows] | MITO.DAO:ENSURE-TABLE-EXISTS
</code></pre>
<p>There are no changes from the previous user definition:</p>
<pre><code class="language-lisp">(mito:migration-expressions 'user)
;=&gt; NIL
</code></pre>
<p>Now let’s add a unique <code>email</code> field:</p>
<pre><code class="language-lisp">(mito:deftable user ()
((name :col-type (:varchar 64))
(email :col-type (:varchar 128)))
(:unique-keys email))
</code></pre>
<p>The migration will run the following code:</p>
<pre><code class="language-lisp">(mito:migration-expressions 'user)
;=&gt; (#&lt;SXQL-STATEMENT: ALTER TABLE user ALTER COLUMN email TYPE character varying(128), ALTER COLUMN email SET NOT NULL&gt;
; #&lt;SXQL-STATEMENT: CREATE UNIQUE INDEX unique_user_email ON user (email)&gt;)
</code></pre>
<p>so let’s apply it:</p>
<pre><code class="language-lisp">(mito:migrate-table 'user)
;-&gt; ;; ALTER TABLE "user" ALTER COLUMN "email" TYPE character varying(128), ALTER COLUMN "email" SET NOT NULL () [0 rows] | MITO.MIGRATION.TABLE:MIGRATE-TABLE
; ;; CREATE UNIQUE INDEX "unique_user_email" ON "user" ("email") () [0 rows] | MITO.MIGRATION.TABLE:MIGRATE-TABLE
;-&gt; (#&lt;SXQL-STATEMENT: ALTER TABLE user ALTER COLUMN email TYPE character varying(128), ALTER COLUMN email SET NOT NULL&gt;
; #&lt;SXQL-STATEMENT: CREATE UNIQUE INDEX unique_user_email ON user (email)&gt;)
</code></pre>
<h3 id="queries">Queries</h3>
<h4 id="creating-objects">Creating objects</h4>
<p>We can create user objects with the regular <code>make-instance</code>:</p>
<pre><code class="language-lisp">(defvar me
(make-instance 'user :name "Eitaro Fukamachi" :email "e.arrows@gmail.com"))
;=&gt; USER
</code></pre>
<p>To save it in DB, use <code>insert-dao</code>:</p>
<pre><code class="language-lisp">(mito:insert-dao me)
;-&gt; ;; INSERT INTO `user` (`name`, `email`, `created_at`, `updated_at`) VALUES (?, ?, ?, ?) ("Eitaro Fukamachi", "e.arrows@gmail.com", "2016-02-04T19:55:16.365543Z", "2016-02-04T19:55:16.365543Z") [0 rows] | MITO.DAO:INSERT-DAO
;=&gt; #&lt;USER {10053C4453}&gt;
</code></pre>
<p>Do the two steps above at once:</p>
<pre><code class="language-lisp">(mito:create-dao 'user :name "Eitaro Fukamachi" :email "e.arrows@gmail.com")
</code></pre>
<p>You should not export the <code>user</code> class and create objects outside of
its package (it is good practice anyway to keep all database-related
operations in say a <code>models</code> package and file). You should instead use
a helper function:</p>
<pre><code class="language-lisp">(defun make-user (&amp;key name)
(make-instance 'user :name name))
</code></pre>
<h4 id="updating-fields">Updating fields</h4>
<pre><code class="language-lisp">(setf (slot-value me 'name) "nitro_idiot")
;=&gt; "nitro_idiot"
</code></pre>
<p>and save it:</p>
<pre><code class="language-lisp">(mito:save-dao me)
</code></pre>
<h4 id="deleting">Deleting</h4>
<pre><code class="language-lisp">(mito:delete-dao me)
;-&gt; ;; DELETE FROM `user` WHERE (`id` = ?) (1) [0 rows] | MITO.DAO:DELETE-DAO
;; or:
(mito:delete-by-values 'user :id 1)
;-&gt; ;; DELETE FROM `user` WHERE (`id` = ?) (1) [0 rows] | MITO.DAO:DELETE-DAO
</code></pre>
<h4 id="get-the-primary-key-value">Get the primary key value</h4>
<pre><code class="language-lisp">(mito:object-id me)
;=&gt; 1
</code></pre>
<h4 id="count">Count</h4>
<pre><code class="language-lisp">(mito:count-dao 'user)
;=&gt; 1
</code></pre>
<h4 id="find-one">Find one</h4>
<pre><code class="language-lisp">(mito:find-dao 'user :id 1)
;-&gt; ;; SELECT * FROM `user` WHERE (`id` = ?) LIMIT 1 (1) [1 row] | MITO.DB:RETRIEVE-BY-SQL
;=&gt; #&lt;USER {10077C6073}&gt;
</code></pre>
<p>So here’s a possibility of generic helpers to find an object by a given key:</p>
<pre><code class="language-lisp">(defgeneric find-user (key-name key-value)
(:documentation "Retrieves an user from the data base by one of the unique
keys."))
(defmethod find-user ((key-name (eql :id)) (key-value integer))
(mito:find-dao 'user key-value))
(defmethod find-user ((key-name (eql :name)) (key-value string))
(first (mito:select-dao 'user
(sxql:where (:= :name key-value)))))
</code></pre>
<h4 id="find-all">Find all</h4>
<p>Use the macro <code>select-dao</code>.</p>
<p>Get a list of all users:</p>
<pre><code class="language-lisp">(mito:select-dao 'user)
;(#&lt;USER {10077C6073}&gt;)
;#&lt;SXQL-STATEMENT: SELECT * FROM user&gt;
</code></pre>
<h4 id="find-by-relationship">Find by relationship</h4>
<p>As seen above:</p>
<pre><code class="language-lisp">(mito:find-dao 'tweet :user *user*)
</code></pre>
<h4 id="custom-queries">Custom queries</h4>
<p>It is with <code>select-dao</code> that you can write more precise queries by
giving it <a href="https://github.com/fukamachi/sxql">SxQL</a> statements.</p>
<p>Example:</p>
<pre><code class="language-lisp">(select-dao 'tweet
(where (:like :status "%Japan%")))
</code></pre>
<p>another:</p>
<pre><code class="language-lisp">(select (:id :name :sex)
(from (:as :person :p))
(where (:and (:&gt;= :age 18)
(:&lt; :age 65)))
(order-by (:desc :age)))
</code></pre>
<p>You can compose your queries with regular Lisp code:</p>
<pre><code class="language-lisp">(defun find-tweets (&amp;key user)
(select-dao 'tweet
(when user
(where (:= :user user)))
(order-by :object-created)))
</code></pre>
<p><code>select-dao</code> is a macro that expands to the right thing©.</p>
<div class="info-box info">
<strong>Note:</strong> if you didn't <code>use</code> SXQL, then write <code>(sxql:where …)</code> and <code>(sxql:order-by …)</code>.
</div>
<p><br /></p>
<p>You can compose your queries further with the backquote syntax.</p>
<p>Imagine you receive a <code>query</code> string, maybe composed of
space-separated words, and you want to search for books that have
either one of these words in their title or in their author’s
name. Searching for “bob adventure” would return a book that has
“adventure” in its title and “bob” in its author name, or both in the
title.</p>
<p>For the example sake, an author is a string, not a link to another table:</p>
<pre><code class="language-lisp">(mito:deftable book ()
((title :col-type (:varchar 128))
(author :col-type (:varchar 128))
(ean :col-type (or (:varchar 128) :null))))
</code></pre>
<p>You want to add a clause that searches on both fields for each word.</p>
<pre><code class="language-lisp">(defun find-books (&amp;key query (order :desc))
"Return a list of books. If a query string is given, search on both the title and the author fields."
(mito:select-dao 'book
(when (str:non-blank-string-p query)
(sxql:where
`(:and
,@(loop for word in (str:words query)
:collect `(:or (:like :title ,(str:concat "%" word "%"))
(:like :authors ,(str:concat "%" word "%")))))))
(sxql:order-by `(,order :created-at))))
</code></pre>
<p>By the way, we are still using a <code>LIKE</code> statement, but with a non-small dataset you’ll want to use your database’s full text search engine.</p>
<h4 id="clauses">Clauses</h4>
<p>See the <a href="https://github.com/fukamachi/sxql#sql-clauses">SxQL documentation</a>.</p>
<p>Examples:</p>
<pre><code class="language-lisp">(select-dao 'foo
(where (:and (:&gt; :age 20) (:&lt;= :age 65))))
</code></pre>
<pre><code class="language-lisp">(order-by :age (:desc :id))
</code></pre>
<pre><code class="language-lisp">(group-by :sex)
</code></pre>
<pre><code class="language-lisp">(having (:&gt;= (:sum :hoge) 88))
</code></pre>
<pre><code class="language-lisp">(limit 0 10)
</code></pre>
<p>and <code>join</code>s, etc.</p>
<h4 id="operators">Operators</h4>
<pre><code class="language-lisp">:not
:is-null, :not-null
:asc, :desc
:distinct
:=, :!=
:&lt;, :&gt;, :&lt;= :&gt;=
:a&lt;, :a&gt;
:as
:in, :not-in
:like
:and, :or
:+, :-, :* :/ :%
:raw
</code></pre>
<h3 id="triggers">Triggers</h3>
<p>Since <code>insert-dao</code>, <code>update-dao</code> and <code>delete-dao</code> are defined as generic
functions, you can define <code>:before</code>, <code>:after</code> or <code>:around</code> methods to those, like regular <a href="clos.html#qualifiers-and-method-combination">method combination</a>.</p>
<pre><code class="language-lisp">(defmethod mito:insert-dao :before ((object user))
(format t "~&amp;Adding ~S...~%" (user-name object)))
(mito:create-dao 'user :name "Eitaro Fukamachi" :email "e.arrows@gmail.com")
;-&gt; Adding "Eitaro Fukamachi"...
; ;; INSERT INTO "user" ("name", "email", "created_at", "updated_at") VALUES (?, ?, ?, ?) ("Eitaro Fukamachi", "e.arrows@gmail.com", "2016-02-16 21:13:47", "2016-02-16 21:13:47") [0 rows] | MITO.DAO:INSERT-DAO
;=&gt; #&lt;USER {100835FB33}&gt;
</code></pre>
<h3 id="inflationdeflation">Inflation/Deflation</h3>
<p>Inflation/Deflation is a function to convert values between Mito and RDBMS.</p>
<pre><code class="language-lisp">(mito:deftable user-report ()
((title :col-type (:varchar 100))
(body :col-type :text
:initform "")
(reported-at :col-type :timestamp
:initform (local-time:now)
:inflate #'local-time:universal-to-timestamp
:deflate #'local-time:timestamp-to-universal)))
</code></pre>
<h3 id="eager-loading">Eager loading</h3>
<p>One of the pains in the neck to use ORMs is the “N+1 query” problem.</p>
<pre><code class="language-lisp">;; BAD EXAMPLE
(use-package '(:mito :sxql))
(defvar *tweets-contain-japan*
(select-dao 'tweet
(where (:like :status "%Japan%"))))
;; Getting names of tweeted users.
(mapcar (lambda (tweet)
(user-name (tweet-user tweet)))
*tweets-contain-japan*)
</code></pre>
<p>This example sends a query to retrieve a user like “SELECT * FROM user
WHERE id = ?” at each iteration.</p>
<p>To prevent this performance issue, add <code>includes</code> to the above query
which only sends a single WHERE IN query instead of N queries:</p>
<pre><code class="language-lisp">;; GOOD EXAMPLE with eager loading
(use-package '(:mito :sxql))
(defvar *tweets-contain-japan*
(select-dao 'tweet
(includes 'user)
(where (:like :status "%Japan%"))))
;-&gt; ;; SELECT * FROM `tweet` WHERE (`status` LIKE ?) ("%Japan%") [3 row] | MITO.DB:RETRIEVE-BY-SQL
;-&gt; ;; SELECT * FROM `user` WHERE (`id` IN (?, ?, ?)) (1, 3, 12) [3 row] | MITO.DB:RETRIEVE-BY-SQL
;=&gt; (#&lt;TWEET {1003513EC3}&gt; #&lt;TWEET {1007BABEF3}&gt; #&lt;TWEET {1007BB9D63}&gt;)
;; No additional SQLs will be executed.
(tweet-user (first *))
;=&gt; #&lt;USER {100361E813}&gt;
</code></pre>
<h3 id="schema-versioning">Schema versioning</h3>
<pre><code>$ ros install mito
$ mito
Usage: mito command [option...]
Commands:
generate-migrations
migrate
Options:
-t, --type DRIVER-TYPE DBI driver type (one of "mysql", "postgres" or "sqlite3")
-d, --database DATABASE-NAME Database name to use
-u, --username USERNAME Username for RDBMS
-p, --password PASSWORD Password for RDBMS
-s, --system SYSTEM ASDF system to load (several -s's allowed)
-D, --directory DIRECTORY Directory path to keep migration SQL files (default: "/Users/nitro_idiot/Programs/lib/mito/db/")
--dry-run List SQL expressions to migrate
</code></pre>
<h3 id="introspection">Introspection</h3>
<p>Mito provides some functions for introspection.</p>
<p>We can access the information of <strong>columns</strong> with the functions in
<code>(mito.class.column:...)</code>:</p>
<ul>
<li><code>table-column-[class, name, info, not-null-p,...]</code></li>
<li><code>primary-key-p</code></li>
</ul>
<p>and likewise for <strong>tables</strong> with <code>(mito.class.table:...)</code>.</p>
<p>Given we get a list of slots of our class:</p>
<pre><code class="language-lisp">(ql:quickload "closer-mop")
(closer-mop:class-direct-slots (find-class 'user))
;; (#&lt;MITO.DAO.COLUMN:DAO-TABLE-COLUMN-CLASS NAME&gt;
;; #&lt;MITO.DAO.COLUMN:DAO-TABLE-COLUMN-CLASS EMAIL&gt;)
(defparameter user-slots *)
</code></pre>
<p>We can answer the following questions:</p>
<h4 id="what-is-the-type-of-this-column-">What is the type of this column ?</h4>
<pre><code class="language-lisp">(mito.class.column:table-column-type (first user-slots))
;; (:VARCHAR 64)
</code></pre>
<h4 id="is-this-column-nullable-">Is this column nullable ?</h4>
<pre><code class="language-lisp">(mito.class.column:table-column-not-null-p
(first user-slots))
;; T
(mito.class.column:table-column-not-null-p
(second user-slots))
;; NIL
</code></pre>
<h3 id="testing">Testing</h3>
<p>We don’t want to test DB operations against the production one. We
need to create a temporary DB before each test.</p>
<p>The macro below creates a temporary DB with a random name, creates the
tables, runs the code and connects back to the original DB connection.</p>
<pre><code class="language-lisp">(defpackage my-test.utils
(:use :cl)
(:import-from :my.models
:*db*
:*db-name*
:connect
:ensure-tables-exist
:migrate-all)
(:export :with-empty-db))
(in-package my-test.utils)
(defun random-string (length)
;; thanks 40ants/hacrm.
(let ((chars "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"))
(coerce (loop repeat length
collect (aref chars (random (length chars))))
'string)))
(defmacro with-empty-db (&amp;body body)
"Run `body` with a new temporary DB."
`(let* ((*random-state* (make-random-state t))
(prefix (concatenate 'string
(random-string 8)
"/"))
;; Save our current DB connection.
(connection mito:*connection*))
(uiop:with-temporary-file (:pathname name :prefix prefix)
;; Bind our *db-name* to a new name, so as to create a new DB.
(let* ((*db-name* name))
;; Always re-connect to our real DB even in case of error in body.
(unwind-protect
(progn
;; our functions to connect to the DB, create the tables and run the migrations.
(connect)
(ensure-tables-exist)
(migrate-all)
,@body)
(setf mito:*connection* connection))))))
</code></pre>
<p>Use it like this:</p>
<pre><code class="language-lisp">(prove:subtest "Creation in a temporary DB."
(with-empty-db
(let ((user (make-user :name "Cookbook")))
(save-user user)
(prove:is (name user)
"Cookbook"
"Test username in a temp DB."))))
;; Creation in a temporary DB
;; CREATE TABLE "user" (
;; id BIGSERIAL NOT NULL PRIMARY KEY,
;; name VARCHAR(64) NOT NULL,
;; email VARCHAR(128) NOT NULL,
;; created_at TIMESTAMP,
;; updated_at TIMESTAMP,
;; UNIQUE (email)
;; ) () [0 rows] | MITO.DB:EXECUTE-SQL
;; ✓ Test username in a temp DB.
</code></pre>
<h2 id="see-also">See also</h2>
<ul>
<li>
<p><a href="https://sites.google.com/site/sabraonthehill/postmodern-examples/exploring-a-database">exploring an existing (PostgreSQL) database with postmodern</a></p>
</li>
<li><a href="https://github.com/fukamachi/mito-attachment">mito-attachment</a></li>
<li><a href="https://github.com/fukamachi/mito-auth">mito-auth</a></li>
<li><a href="https://github.com/fukamachi/can/">can</a> a role-based access right control library</li>
<li>an advanced <a href="drafts/defmodel.lisp.html">“defmodel” macro</a>.</li>
</ul>
<!-- # todo: Generating models for an existing DB -->
<p class="page-source">
Page source: <a href="https://github.com/LispCookbook/cl-cookbook/blob/master/databases.md">databases.md</a>
</p>
</div>
<script type="text/javascript">
// Don't write the TOC on the index.
if (window.location.pathname != "/cl-cookbook/") {
$("#toc").toc({
content: "#content", // will ignore the first h1 with the site+page title.
headings: "h1,h2,h3,h4"});
}
$("#two-cols + ul").css({
"column-count": "2",
});
$("#contributors + ul").css({
"column-count": "4",
});
</script>
<div>
<footer class="footer">
<hr/>
&copy; 2002&ndash;2021 the Common Lisp Cookbook Project
</footer>
</div>
<div id="toc-btn">T<br>O<br>C</div>
</div>
<script text="javascript">
HighlightLisp.highlight_auto({className: null});
</script>
<script type="text/javascript">
function duckSearch() {
var searchField = document.getElementById("searchField");
if (searchField && searchField.value) {
var query = escape("site:lispcookbook.github.io/cl-cookbook/ " + searchField.value);
window.location.href = "https://duckduckgo.com/?kj=b2&kf=-1&ko=1&q=" + query;
// https://duckduckgo.com/params
// kj=b2: blue header in results page
// kf=-1: no favicons
}
}
</script>
<script async defer data-domain="lispcookbook.github.io/cl-cookbook" src="https://plausible.io/js/plausible.js"></script>
</body>
</html>
Reference in a new issue Copy permalink