2022-08-02 12:34:59 +02:00
<!DOCTYPE html>
< html lang = "en" >
< head >
< meta name = "generator" content =
"HTML Tidy for HTML5 for Linux version 5.2.0">
< title > Database Access and Persistence< / title >
< meta charset = "utf-8" >
< meta name = "description" content = "A collection of examples of using Common Lisp" >
< meta name = "viewport" content =
"width=device-width, initial-scale=1">
2022-08-04 11:37:48 +02:00
< link rel = "icon" href =
"assets/cl-logo-blue.png"/>
2022-08-02 12:34:59 +02:00
< link rel = "stylesheet" href =
"assets/style.css">
< script type = "text/javascript" src =
"assets/highlight-lisp.js">
< / script >
< script type = "text/javascript" src =
"assets/jquery-3.2.1.min.js">
< / script >
< script type = "text/javascript" src =
"assets/jquery.toc/jquery.toc.min.js">
< / script >
< script type = "text/javascript" src =
"assets/toggle-toc.js">
< / script >
< link rel = "stylesheet" href =
"assets/github.css">
< link rel = "stylesheet" href = "https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity = "sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin = "anonymous" >
< / head >
< body >
< h1 id = "title-xs" > < a href = "index.html" > The Common Lisp Cookbook< / a > – Database Access and Persistence< / h1 >
< div id = "logo-container" >
< a href = "index.html" >
< img id = "logo" src = "assets/cl-logo-blue.png" / >
< / a >
< div id = "searchform-container" >
< form onsubmit = "duckSearch()" action = "javascript:void(0)" >
< input id = "searchField" type = "text" value = "" placeholder = "Search..." >
< / form >
< / div >
< div id = "toc-container" class = "toc-close" >
< div id = "toc-title" > Table of Contents< / div >
< ul id = "toc" class = "list-unstyled" > < / ul >
< / div >
< / div >
< div id = "content-container" >
< h1 id = "title-non-xs" > < a href = "index.html" > The Common Lisp Cookbook< / a > – Database Access and Persistence< / h1 >
<!-- Announcement we can keep for 1 month or more. I remove it and re - add it from time to time. -->
< p class = "announce" >
📹 < a href = "https://www.udemy.com/course/common-lisp-programming/?couponCode=6926D599AA-LISP4ALL" > NEW! Learn Lisp in videos and support our contributors with this 40% discount.< / a >
< / p >
< p class = "announce-neutral" >
📕 < a href = "index.html#download-in-epub" > Get the EPUB and PDF< / a >
< / p >
< div id = "content"
< p > The
< a href = "https://github.com/CodyReichert/awesome-cl#database" > Database section on the Awesome-cl list< / a >
is a resource listing popular libraries to work with different kind of
databases. We can group them roughly in four categories:< / p >
< ul >
< li > wrappers to one database engine (cl-sqlite, postmodern, cl-redis,…),< / li >
< li > interfaces to several DB engines (clsql, sxql,…),< / li >
< li > persistent object databases (bknr.datastore (see chap. 21 of “Common Lisp Recipes”), ubiquitous,…),< / li >
< li > < a href = "https://en.wikipedia.org/wiki/Object-relational_mapping" > Object Relational Mappers< / a > (Mito),< / li >
< / ul >
< p > and other DB-related tools (pgloader).< / p >
< p > We’
existing DB, you might want to have a look at cl-dbi and clsql. If you
don’
objects, you also have a choice of libraries.< / p >
< h2 id = "the-mito-orm-and-sxql" > The Mito ORM and SxQL< / h2 >
< p > Mito is in Quicklisp:< / p >
< pre > < code class = "language-lisp" > (ql:quickload "mito")
< / code > < / pre >
< h3 id = "overview" > Overview< / h3 >
< p > < a href = "https://github.com/fukamachi/mito" > Mito< / a > is “an ORM for Common Lisp
with migrations, relationships and PostgreSQL support”.< / p >
< ul >
< li > it < strong > supports MySQL, PostgreSQL and SQLite3< / strong > ,< / li >
< li > when defining a model, it adds an < code > id< / code > (serial primary key),
< code > created_at< / code > and < code > updated_at< / code > fields by default like Ruby’
ActiveRecord or Django,< / li >
< li > handles DB < strong > migrations< / strong > for the supported backends,< / li >
< li > permits DB < strong > schema versioning< / strong > ,< / li >
< li > is tested under SBCL and CCL.< / li >
< / ul >
< p > As an ORM, it allows to write class definitions, to specify relationships, and
provides functions to query the database. For custom queries, it relies on
< a href = "https://github.com/fukamachi/sxql" > SxQL< / a > , an SQL generator that provides the
same interface for several backends.< / p >
< p > Working with Mito generally involves these steps:< / p >
< ul >
< li > connecting to the DB< / li >
< li > writing < a href = "clos.html" > CLOS< / a > classes to define models< / li >
< li > running migrations to create or alter tables< / li >
< li > creating objects, saving same in the DB,< / li >
< / ul >
< p > and iterating.< / p >
< h3 id = "connecting-to-a-db" > Connecting to a DB< / h3 >
< p > Mito provides the function < code > connect-toplevel< / code > to establish a
connection to RDBMs:< / p >
< pre > < code class = "language-lisp" > (mito:connect-toplevel :mysql :database-name "myapp" :username "fukamachi" :password "c0mon-1isp")
< / code > < / pre >
< p > The driver type can be of < code > :mysql< / code > , < code > :sqlite3< / code > and < code > :postgres< / code > .< / p >
< p > With sqlite you don’ < / p >
< pre > < code class = "language-lisp" > (mito:connect-toplevel :sqlite3 :database-name "myapp")
< / code > < / pre >
< p > As usual, you need to create the MySQL or PostgreSQL database beforehand.
Refer to their documentation.< / p >
< p > Connecting sets < code > mito:*connection*< / code > to the new connection and returns it.< / p >
< p > Disconnect with < code > disconnect-toplevel< / code > .< / p >
< p > You might make good use of a wrapper function:< / p >
< pre > < code class = "language-lisp" > (defun connect ()
"Connect to the DB."
(mito:connect-toplevel :sqlite3 :database-name "myapp"))
< / code > < / pre >
< h3 id = "models" > Models< / h3 >
< h4 id = "defining-models" > Defining models< / h4 >
< p > In Mito, you can define a class which corresponds to a database table with the < code > deftable< / code > macro:< / p >
< pre > < code class = "language-lisp" > (mito:deftable user ()
((name :col-type (:varchar 64))
(email :col-type (or (:varchar 128) :null)))
< / code > < / pre >
< p > Alternatively, you can specify < code > (:metaclass mito:dao-table-class)< / code > in a regular class definition.< / p >
< p > The < code > deftable< / code > macro automatically adds some slots: a primary key named < code > id< / code > if there’ < code > created_at< / code > and < code > updated_at< / code > for recording timestamps. Specifying < code > (:auto-pk nil)< / code > and < code > (:record-timestamps nil)< / code > in the < code > deftable< / code > form will disable these behaviours. A < code > deftable< / code > class will also come with initializers, named after the slot, and accessors, of form < code > < class-name> -< slot-name> < / code > , for each named slot. For example, for the < code > name< / code > slot in the above table definition, the initarg < code > :name< / code > will be added to the constuctor, and the accessor < code > user-name< / code > will be created.< / p >
< p > You can inspect the new class:< / p >
< pre > < code class = "language-lisp" > (mito.class:table-column-slots (find-class 'user))
;=> (#< MITO.DAO.COLUMN:DAO-TABLE-COLUMN-CLASS MITO.DAO.MIXIN::ID>
; #< MITO.DAO.COLUMN:DAO-TABLE-COLUMN-CLASS COMMON-LISP-USER::NAME>
; #< MITO.DAO.COLUMN:DAO-TABLE-COLUMN-CLASS COMMON-LISP-USER::EMAIL>
; #< MITO.DAO.COLUMN:DAO-TABLE-COLUMN-CLASS MITO.DAO.MIXIN::CREATED-AT>
; #< MITO.DAO.COLUMN:DAO-TABLE-COLUMN-CLASS MITO.DAO.MIXIN::UPDATED-AT> )
< / code > < / pre >
< p > The class inherits < code > mito:dao-class< / code > implicitly.< / p >
< pre > < code class = "language-lisp" > (find-class 'user)
;=> #< MITO.DAO.TABLE:DAO-TABLE-CLASS COMMON-LISP-USER::USER>
(c2mop:class-direct-superclasses *)
;=> (#< STANDARD-CLASS MITO.DAO.TABLE:DAO-CLASS> )
< / code > < / pre >
< p > This may be useful when you define methods which can be applied for
all table classes.< / p >
< p > For more information on using the Common Lisp Object System, see the
< a href = "clos.html" > clos< / a > page.< / p >
< h4 id = "creating-the-tables" > Creating the tables< / h4 >
< p > After defining the models, you must create the tables:< / p >
< pre > < code class = "language-lisp" > (mito:ensure-table-exists 'user)
< / code > < / pre >
< p > So a helper function:< / p >
< pre > < code class = "language-lisp" > (defun ensure-tables ()
(mapcar #'mito:ensure-table-exists '(user foo bar)))
< / code > < / pre >
< p > See
< a href = "https://github.com/fukamachi/mito#generating-table-definitions" > Mito’ < / a >
for a couple more ways.< / p >
< p > When you alter the model you’ < / p >
< h4 id = "fields" > Fields< / h4 >
< h5 id = "fields-types" > Fields types< / h5 >
< p > Field types are:< / p >
< p > < code > (:varchar < integer> )< / code > ,< / p >
< p > < code > :serial< / code > , < code > :bigserial< / code > , < code > :integer< / code > , < code > :bigint< / code > , < code > :unsigned< / code > ,< / p >
< p > < code > :timestamp< / code > , < code > :timestamptz< / code > ,< / p >
< p > < code > :bytea< / code > ,< / p >
< h5 id = "optional-fields" > Optional fields< / h5 >
< p > Use < code > (or < real type> :null)< / code > :< / p >
< pre > < code class = "language-lisp" > (email :col-type (or (:varchar 128) :null))
< / code > < / pre >
< h5 id = "field-constraints" > Field constraints< / h5 >
< p > < code > :unique-keys< / code > can be used like so:< / p >
< pre > < code class = "language-lisp" > (mito:deftable user ()
((name :col-type (:varchar 64))
(email :col-type (:varchar 128))
(:unique-keys email))
< / code > < / pre >
< p > We already saw < code > :primary-key< / code > .< / p >
< p > You can change the table name with < code > :table-name< / code > .< / p >
< h4 id = "relationships" > Relationships< / h4 >
< p > You can define a relationship by specifying a foreign class with < code > :col-type< / code > :< / p >
< pre > < code class = "language-lisp" > (mito:deftable tweet ()
((status :col-type :text)
;; This slot refers to USER class
(user :col-type user))
(table-definition (find-class 'tweet))
;=> (#< SXQL-STATEMENT: CREATE TABLE tweet (
; id BIGSERIAL NOT NULL PRIMARY KEY,
; status TEXT NOT NULL,
; user_id BIGINT NOT NULL,
; created_at TIMESTAMP,
; updated_at TIMESTAMP
; )> )
< / code > < / pre >
< p > Now you can create or retrieve a < code > TWEET< / code > by a < code > USER< / code > object, not a < code > USER-ID< / code > .< / p >
< pre > < code class = "language-lisp" > (defvar *user* (mito:create-dao 'user :name "Eitaro Fukamachi"))
(mito:create-dao 'tweet :user *user*)
(mito:find-dao 'tweet :user *user*)
< / code > < / pre >
< p > Mito doesn’ < / p >
< h5 id = "one-to-one" > One-to-one< / h5 >
< p > A one-to-one relationship is simply represented with a simple foreign
key on a slot (as < code > :col-type user< / code > in the < code > tweet< / code > class). Besides, we
can add a unicity constraint, as with < code > (:unique-keys email)< / code > .< / p >
< h5 id = "one-to-many-many-to-one" > One-to-many, many-to-one< / h5 >
< p > The tweet example above shows a one-to-many relationship between a user and
his tweets: a user can write many tweets, and a tweet belongs to only
one user.< / p >
< p > The relationship is defined with a foreign key on the “many” side
linking back to the “one” side. Here the < code > tweet< / code > class defines a
< code > user< / code > foreign key, so a tweet can only have one user. You didn’
to edit the < code > user< / code > class.< / p >
< p > A many-to-one relationship is actually the contrary of a one-to-many.
You have to put the foreign key on the appropriate side.< / p >
< h5 id = "many-to-many" > Many-to-many< / h5 >
< p > A many-to-many relationship needs an intermediate table, which will be
the “many” side for the two tables it is the intermediary of.< / p >
< p > And, thanks to the join table, we can store more information about the relationship.< / p >
< p > Let’ < code > book< / code > class:< / p >
< pre > < code class = "language-lisp" > (mito:deftable book ()
((title :col-type (:varchar 128))
(ean :col-type (or (:varchar 128) :null))))
< / code > < / pre >
< p > A user can have many books, and a book (as the title, not the physical
copy) is likely to be in many people’ ’
intermediate class:< / p >
< pre > < code class = "language-lisp" > (mito:deftable user-books ()
((user :col-type user)
(book :col-type book)))
< / code > < / pre >
< p > Each time we want to add a book to a user’
a < code > add-book< / code > function), we create a new < code > user-books< / code > object.< / p >
< p > But someone may very well own many copies of one book. This is an
information we can store in the join table:< / p >
< pre > < code class = "language-lisp" > (mito:deftable user-books ()
((user :col-type user)
(book :col-type book)
;; Set the quantity, 1 by default:
(quantity :col-type :integer)))
< / code > < / pre >
< h4 id = "inheritance-and-mixin" > Inheritance and mixin< / h4 >
< p > A subclass of DAO-CLASS is allowed to be inherited. This may be useful
when you need classes which have similar columns:< / p >
< pre > < code class = "language-lisp" > (mito:deftable user ()
((name :col-type (:varchar 64))
(email :col-type (:varchar 128)))
(:unique-keys email))
(mito:deftable temporary-user (user)
((registered-at :col-type :timestamp)))
(mito:table-definition 'temporary-user)
;=> (#< SXQL-STATEMENT: CREATE TABLE temporary_user (
; id BIGSERIAL NOT NULL PRIMARY KEY,
; name VARCHAR(64) NOT NULL,
; email VARCHAR(128) NOT NULL,
; registered_at TIMESTAMP NOT NULL,
; created_at TIMESTAMP,
; updated_at TIMESTAMP,
; UNIQUE (email)
; )> )
< / code > < / pre >
< p > If you need a ‘ ’ ’
database tables, you can use < code > DAO-TABLE-MIXIN< / code > in a < code > defclass< / code > form. The < code > has-email< / code >
class below will not create a table.< / p >
< pre > < code class = "language-lisp" > (defclass has-email ()
((email :col-type (:varchar 128)
:initarg :email
:accessor object-email))
(:metaclass mito:dao-table-mixin)
(:unique-keys email))
;=> #< MITO.DAO.MIXIN:DAO-TABLE-MIXIN COMMON-LISP-USER::HAS-EMAIL>
(mito:deftable user (has-email)
((name :col-type (:varchar 64))))
;=> #< MITO.DAO.TABLE:DAO-TABLE-CLASS COMMON-LISP-USER::USER>
(mito:table-definition 'user)
;=> (#< SXQL-STATEMENT: CREATE TABLE user (
; id BIGSERIAL NOT NULL PRIMARY KEY,
; name VARCHAR(64) NOT NULL,
; email VARCHAR(128) NOT NULL,
; created_at TIMESTAMP,
; updated_at TIMESTAMP,
; UNIQUE (email)
; )> )
< / code > < / pre >
< p > See more examples of use in < a href = "https://github.com/fukamachi/mito-auth/" > mito-auth< / a > .< / p >
< h4 id = "troubleshooting" > Troubleshooting< / h4 >
< h5 id = "cannot-change-class-objects-into-class-metaobjects" > “Cannot CHANGE-CLASS objects into CLASS metaobjects.”< / h5 >
< p > If you get the following error message:< / p >
< pre > < code > Cannot CHANGE-CLASS objects into CLASS metaobjects.
[Condition of type SB-PCL::METAOBJECT-INITIALIZATION-VIOLATION]
See also:
The Art of the Metaobject Protocol, CLASS [:initialization]
< / code > < / pre >
< p > it is certainly because you first wrote a class definition and < em > then< / em >
added the Mito metaclass and tried to evaluate the class definition
again.< / p >
< p > If this happens, you must remove the class definition from the current package:< / p >
< pre > < code class = "language-lisp" > (setf (find-class 'foo) nil)
< / code > < / pre >
< p > or, with the Slime inspector, click on the class and find the “remove” button.< / p >
< p > More info < a href = "https://stackoverflow.com/questions/38811931/how-to-change-classs-metaclass" > here< / a > .< / p >
< h3 id = "migrations" > Migrations< / h3 >
< p > We can run database migrations manually, as shown below, or we can
automatically run migrations after a change to the model
definitions. To enable automatic migrations, set < code > mito:*auto-migration-mode*< / code > to < code > t< / code > .< / p >
< p > The first step is to create the tables, if needed:< / p >
< pre > < code class = "language-lisp" > (ensure-table-exists 'user)
< / code > < / pre >
< p > then alter the tables:< / p >
< pre > < code class = "language-lisp" > (mito:migrate-table 'user)
< / code > < / pre >
< p > You can check the SQL generated code with < code > migration-expressions
'class< / code > . For example, we create the < code > user< / code > table:< / p >
< pre > < code class = "language-lisp" > (ensure-table-exists 'user)
;-> ;; CREATE TABLE IF NOT EXISTS "user" (
; "id" BIGSERIAL NOT NULL PRIMARY KEY,
; "name" VARCHAR(64) NOT NULL,
; "email" VARCHAR(128),
; "created_at" TIMESTAMP,
; "updated_at" TIMESTAMP
; ) () [0 rows] | MITO.DAO:ENSURE-TABLE-EXISTS
< / code > < / pre >
< p > There are no changes from the previous user definition:< / p >
< pre > < code class = "language-lisp" > (mito:migration-expressions 'user)
;=> NIL
< / code > < / pre >
< p > Now let’ < code > email< / code > field:< / p >
< pre > < code class = "language-lisp" > (mito:deftable user ()
((name :col-type (:varchar 64))
(email :col-type (:varchar 128)))
(:unique-keys email))
< / code > < / pre >
< p > The migration will run the following code:< / p >
< pre > < code class = "language-lisp" > (mito:migration-expressions 'user)
;=> (#< SXQL-STATEMENT: ALTER TABLE user ALTER COLUMN email TYPE character varying(128), ALTER COLUMN email SET NOT NULL>
; #< SXQL-STATEMENT: CREATE UNIQUE INDEX unique_user_email ON user (email)> )
< / code > < / pre >
< p > so let’ < / p >
< pre > < code class = "language-lisp" > (mito:migrate-table 'user)
;-> ;; ALTER TABLE "user" ALTER COLUMN "email" TYPE character varying(128), ALTER COLUMN "email" SET NOT NULL () [0 rows] | MITO.MIGRATION.TABLE:MIGRATE-TABLE
; ;; CREATE UNIQUE INDEX "unique_user_email" ON "user" ("email") () [0 rows] | MITO.MIGRATION.TABLE:MIGRATE-TABLE
;-> (#< SXQL-STATEMENT: ALTER TABLE user ALTER COLUMN email TYPE character varying(128), ALTER COLUMN email SET NOT NULL>
; #< SXQL-STATEMENT: CREATE UNIQUE INDEX unique_user_email ON user (email)> )
< / code > < / pre >
< h3 id = "queries" > Queries< / h3 >
< h4 id = "creating-objects" > Creating objects< / h4 >
< p > We can create user objects with the regular < code > make-instance< / code > :< / p >
< pre > < code class = "language-lisp" > (defvar me
(make-instance 'user :name "Eitaro Fukamachi" :email "e.arrows@gmail.com"))
;=> USER
< / code > < / pre >
< p > To save it in DB, use < code > insert-dao< / code > :< / p >
< pre > < code class = "language-lisp" > (mito:insert-dao me)
;-> ;; INSERT INTO `user` (`name`, `email`, `created_at`, `updated_at`) VALUES (?, ?, ?, ?) ("Eitaro Fukamachi", "e.arrows@gmail.com", "2016-02-04T19:55:16.365543Z", "2016-02-04T19:55:16.365543Z") [0 rows] | MITO.DAO:INSERT-DAO
;=> #< USER {10053C4453}>
< / code > < / pre >
< p > Do the two steps above at once:< / p >
< pre > < code class = "language-lisp" > (mito:create-dao 'user :name "Eitaro Fukamachi" :email "e.arrows@gmail.com")
< / code > < / pre >
< p > You should not export the < code > user< / code > class and create objects outside of
its package (it is good practice anyway to keep all database-related
operations in say a < code > models< / code > package and file). You should instead use
a helper function:< / p >
< pre > < code class = "language-lisp" > (defun make-user (& key name)
(make-instance 'user :name name))
< / code > < / pre >
< h4 id = "updating-fields" > Updating fields< / h4 >
< pre > < code class = "language-lisp" > (setf (slot-value me 'name) "nitro_idiot")
;=> "nitro_idiot"
< / code > < / pre >
< p > and save it:< / p >
< pre > < code class = "language-lisp" > (mito:save-dao me)
< / code > < / pre >
< h4 id = "deleting" > Deleting< / h4 >
< pre > < code class = "language-lisp" > (mito:delete-dao me)
;-> ;; DELETE FROM `user` WHERE (`id` = ?) (1) [0 rows] | MITO.DAO:DELETE-DAO
;; or:
(mito:delete-by-values 'user :id 1)
;-> ;; DELETE FROM `user` WHERE (`id` = ?) (1) [0 rows] | MITO.DAO:DELETE-DAO
< / code > < / pre >
< h4 id = "get-the-primary-key-value" > Get the primary key value< / h4 >
< pre > < code class = "language-lisp" > (mito:object-id me)
;=> 1
< / code > < / pre >
< h4 id = "count" > Count< / h4 >
< pre > < code class = "language-lisp" > (mito:count-dao 'user)
;=> 1
< / code > < / pre >
< h4 id = "find-one" > Find one< / h4 >
< pre > < code class = "language-lisp" > (mito:find-dao 'user :id 1)
;-> ;; SELECT * FROM `user` WHERE (`id` = ?) LIMIT 1 (1) [1 row] | MITO.DB:RETRIEVE-BY-SQL
;=> #< USER {10077C6073}>
< / code > < / pre >
< p > So here’ < / p >
< pre > < code class = "language-lisp" > (defgeneric find-user (key-name key-value)
(:documentation "Retrieves an user from the data base by one of the unique
keys."))
(defmethod find-user ((key-name (eql :id)) (key-value integer))
(mito:find-dao 'user key-value))
(defmethod find-user ((key-name (eql :name)) (key-value string))
(first (mito:select-dao 'user
(sxql:where (:= :name key-value)))))
< / code > < / pre >
< h4 id = "find-all" > Find all< / h4 >
< p > Use the macro < code > select-dao< / code > .< / p >
< p > Get a list of all users:< / p >
< pre > < code class = "language-lisp" > (mito:select-dao 'user)
;(#< USER {10077C6073}> )
;#< SXQL-STATEMENT: SELECT * FROM user>
< / code > < / pre >
< h4 id = "find-by-relationship" > Find by relationship< / h4 >
< p > As seen above:< / p >
< pre > < code class = "language-lisp" > (mito:find-dao 'tweet :user *user*)
< / code > < / pre >
< h4 id = "custom-queries" > Custom queries< / h4 >
< p > It is with < code > select-dao< / code > that you can write more precise queries by
giving it < a href = "https://github.com/fukamachi/sxql" > SxQL< / a > statements.< / p >
< p > Example:< / p >
< pre > < code class = "language-lisp" > (select-dao 'tweet
(where (:like :status "%Japan%")))
< / code > < / pre >
< p > another:< / p >
< pre > < code class = "language-lisp" > (select (:id :name :sex)
(from (:as :person :p))
(where (:and (:> = :age 18)
(:< :age 65)))
(order-by (:desc :age)))
< / code > < / pre >
< p > You can compose your queries with regular Lisp code:< / p >
< pre > < code class = "language-lisp" > (defun find-tweets (& key user)
(select-dao 'tweet
(when user
(where (:= :user user)))
(order-by :object-created)))
< / code > < / pre >
< p > < code > select-dao< / code > is a macro that expands to the right thing©.< / p >
< div class = "info-box info" >
< strong > Note:< / strong > if you didn't < code > use< / code > SXQL, then write < code > (sxql:where …)< / code > and < code > (sxql:order-by …)< / code > .
< / div >
< p > < br / > < / p >
< p > You can compose your queries further with the backquote syntax.< / p >
< p > Imagine you receive a < code > query< / code > string, maybe composed of
space-separated words, and you want to search for books that have
either one of these words in their title or in their author’
name. Searching for “bob adventure” would return a book that has
“adventure” in its title and “bob” in its author name, or both in the
title.< / p >
< p > For the example sake, an author is a string, not a link to another table:< / p >
< pre > < code class = "language-lisp" > (mito:deftable book ()
((title :col-type (:varchar 128))
(author :col-type (:varchar 128))
(ean :col-type (or (:varchar 128) :null))))
< / code > < / pre >
< p > You want to add a clause that searches on both fields for each word.< / p >
< pre > < code class = "language-lisp" > (defun find-books (& key query (order :desc))
"Return a list of books. If a query string is given, search on both the title and the author fields."
(mito:select-dao 'book
(when (str:non-blank-string-p query)
(sxql:where
`(:and
,@(loop for word in (str:words query)
:collect `(:or (:like :title ,(str:concat "%" word "%"))
(:like :authors ,(str:concat "%" word "%")))))))
(sxql:order-by `(,order :created-at))))
< / code > < / pre >
< p > By the way, we are still using a < code > LIKE< / code > statement, but with a non-small dataset you’ ’ < / p >
< h4 id = "clauses" > Clauses< / h4 >
< p > See the < a href = "https://github.com/fukamachi/sxql#sql-clauses" > SxQL documentation< / a > .< / p >
< p > Examples:< / p >
< pre > < code class = "language-lisp" > (select-dao 'foo
(where (:and (:> :age 20) (:< = :age 65))))
< / code > < / pre >
< pre > < code class = "language-lisp" > (order-by :age (:desc :id))
< / code > < / pre >
< pre > < code class = "language-lisp" > (group-by :sex)
< / code > < / pre >
< pre > < code class = "language-lisp" > (having (:> = (:sum :hoge) 88))
< / code > < / pre >
< pre > < code class = "language-lisp" > (limit 0 10)
< / code > < / pre >
< p > and < code > join< / code > s, etc.< / p >
< h4 id = "operators" > Operators< / h4 >
< pre > < code class = "language-lisp" > :not
:is-null, :not-null
:asc, :desc
:distinct
:=, :!=
:< , :> , :< = :> =
:a< , :a>
:as
:in, :not-in
:like
:and, :or
:+, :-, :* :/ :%
:raw
< / code > < / pre >
< h3 id = "triggers" > Triggers< / h3 >
< p > Since < code > insert-dao< / code > , < code > update-dao< / code > and < code > delete-dao< / code > are defined as generic
functions, you can define < code > :before< / code > , < code > :after< / code > or < code > :around< / code > methods to those, like regular < a href = "clos.html#qualifiers-and-method-combination" > method combination< / a > .< / p >
< pre > < code class = "language-lisp" > (defmethod mito:insert-dao :before ((object user))
(format t "~& Adding ~S...~%" (user-name object)))
(mito:create-dao 'user :name "Eitaro Fukamachi" :email "e.arrows@gmail.com")
;-> Adding "Eitaro Fukamachi"...
; ;; INSERT INTO "user" ("name", "email", "created_at", "updated_at") VALUES (?, ?, ?, ?) ("Eitaro Fukamachi", "e.arrows@gmail.com", "2016-02-16 21:13:47", "2016-02-16 21:13:47") [0 rows] | MITO.DAO:INSERT-DAO
;=> #< USER {100835FB33}>
< / code > < / pre >
< h3 id = "inflationdeflation" > Inflation/Deflation< / h3 >
< p > Inflation/Deflation is a function to convert values between Mito and RDBMS.< / p >
< pre > < code class = "language-lisp" > (mito:deftable user-report ()
((title :col-type (:varchar 100))
(body :col-type :text
:initform "")
(reported-at :col-type :timestamp
:initform (local-time:now)
:inflate #'local-time:universal-to-timestamp
:deflate #'local-time:timestamp-to-universal)))
< / code > < / pre >
< h3 id = "eager-loading" > Eager loading< / h3 >
< p > One of the pains in the neck to use ORMs is the “N+1 query” problem.< / p >
< pre > < code class = "language-lisp" > ;; BAD EXAMPLE
(use-package '(:mito :sxql))
(defvar *tweets-contain-japan*
(select-dao 'tweet
(where (:like :status "%Japan%"))))
;; Getting names of tweeted users.
(mapcar (lambda (tweet)
(user-name (tweet-user tweet)))
*tweets-contain-japan*)
< / code > < / pre >
< p > This example sends a query to retrieve a user like “SELECT * FROM user
WHERE id = ?” at each iteration.< / p >
< p > To prevent this performance issue, add < code > includes< / code > to the above query
which only sends a single WHERE IN query instead of N queries:< / p >
< pre > < code class = "language-lisp" > ;; GOOD EXAMPLE with eager loading
(use-package '(:mito :sxql))
(defvar *tweets-contain-japan*
(select-dao 'tweet
(includes 'user)
(where (:like :status "%Japan%"))))
;-> ;; SELECT * FROM `tweet` WHERE (`status` LIKE ?) ("%Japan%") [3 row] | MITO.DB:RETRIEVE-BY-SQL
;-> ;; SELECT * FROM `user` WHERE (`id` IN (?, ?, ?)) (1, 3, 12) [3 row] | MITO.DB:RETRIEVE-BY-SQL
;=> (#< TWEET {1003513EC3}> #< TWEET {1007BABEF3}> #< TWEET {1007BB9D63}> )
;; No additional SQLs will be executed.
(tweet-user (first *))
;=> #< USER {100361E813}>
< / code > < / pre >
< h3 id = "schema-versioning" > Schema versioning< / h3 >
< pre > < code > $ ros install mito
$ mito
Usage: mito command [option...]
Commands:
generate-migrations
migrate
Options:
-t, --type DRIVER-TYPE DBI driver type (one of "mysql", "postgres" or "sqlite3")
-d, --database DATABASE-NAME Database name to use
-u, --username USERNAME Username for RDBMS
-p, --password PASSWORD Password for RDBMS
-s, --system SYSTEM ASDF system to load (several -s's allowed)
-D, --directory DIRECTORY Directory path to keep migration SQL files (default: "/Users/nitro_idiot/Programs/lib/mito/db/")
--dry-run List SQL expressions to migrate
< / code > < / pre >
< h3 id = "introspection" > Introspection< / h3 >
< p > Mito provides some functions for introspection.< / p >
< p > We can access the information of < strong > columns< / strong > with the functions in
< code > (mito.class.column:...)< / code > :< / p >
< ul >
< li > < code > table-column-[class, name, info, not-null-p,...]< / code > < / li >
< li > < code > primary-key-p< / code > < / li >
< / ul >
< p > and likewise for < strong > tables< / strong > with < code > (mito.class.table:...)< / code > .< / p >
< p > Given we get a list of slots of our class:< / p >
< pre > < code class = "language-lisp" > (ql:quickload "closer-mop")
(closer-mop:class-direct-slots (find-class 'user))
;; (#< MITO.DAO.COLUMN:DAO-TABLE-COLUMN-CLASS NAME>
;; #< MITO.DAO.COLUMN:DAO-TABLE-COLUMN-CLASS EMAIL> )
(defparameter user-slots *)
< / code > < / pre >
< p > We can answer the following questions:< / p >
< h4 id = "what-is-the-type-of-this-column-" > What is the type of this column ?< / h4 >
< pre > < code class = "language-lisp" > (mito.class.column:table-column-type (first user-slots))
;; (:VARCHAR 64)
< / code > < / pre >
< h4 id = "is-this-column-nullable-" > Is this column nullable ?< / h4 >
< pre > < code class = "language-lisp" > (mito.class.column:table-column-not-null-p
(first user-slots))
;; T
(mito.class.column:table-column-not-null-p
(second user-slots))
;; NIL
< / code > < / pre >
< h3 id = "testing" > Testing< / h3 >
< p > We don’
need to create a temporary DB before each test.< / p >
< p > The macro below creates a temporary DB with a random name, creates the
tables, runs the code and connects back to the original DB connection.< / p >
< pre > < code class = "language-lisp" > (defpackage my-test.utils
(:use :cl)
(:import-from :my.models
:*db*
:*db-name*
:connect
:ensure-tables-exist
:migrate-all)
(:export :with-empty-db))
(in-package my-test.utils)
(defun random-string (length)
;; thanks 40ants/hacrm.
(let ((chars "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"))
(coerce (loop repeat length
collect (aref chars (random (length chars))))
'string)))
(defmacro with-empty-db (& body body)
"Run `body` with a new temporary DB."
`(let* ((*random-state* (make-random-state t))
(prefix (concatenate 'string
(random-string 8)
"/"))
;; Save our current DB connection.
(connection mito:*connection*))
(uiop:with-temporary-file (:pathname name :prefix prefix)
;; Bind our *db-name* to a new name, so as to create a new DB.
(let* ((*db-name* name))
;; Always re-connect to our real DB even in case of error in body.
(unwind-protect
(progn
;; our functions to connect to the DB, create the tables and run the migrations.
(connect)
(ensure-tables-exist)
(migrate-all)
,@body)
(setf mito:*connection* connection))))))
< / code > < / pre >
< p > Use it like this:< / p >
< pre > < code class = "language-lisp" > (prove:subtest "Creation in a temporary DB."
(with-empty-db
(let ((user (make-user :name "Cookbook")))
(save-user user)
(prove:is (name user)
"Cookbook"
"Test username in a temp DB."))))
;; Creation in a temporary DB
;; CREATE TABLE "user" (
;; id BIGSERIAL NOT NULL PRIMARY KEY,
;; name VARCHAR(64) NOT NULL,
;; email VARCHAR(128) NOT NULL,
;; created_at TIMESTAMP,
;; updated_at TIMESTAMP,
;; UNIQUE (email)
;; ) () [0 rows] | MITO.DB:EXECUTE-SQL
;; ✓ Test username in a temp DB.
< / code > < / pre >
< h2 id = "see-also" > See also< / h2 >
< ul >
< li >
< p > < a href = "https://sites.google.com/site/sabraonthehill/postmodern-examples/exploring-a-database" > exploring an existing (PostgreSQL) database with postmodern< / a > < / p >
< / li >
< li > < a href = "https://github.com/fukamachi/mito-attachment" > mito-attachment< / a > < / li >
< li > < a href = "https://github.com/fukamachi/mito-auth" > mito-auth< / a > < / li >
< li > < a href = "https://github.com/fukamachi/can/" > can< / a > a role-based access right control library< / li >
< li > an advanced < a href = "drafts/defmodel.lisp.html" > “defmodel” macro< / a > .< / li >
< / ul >
<!-- # todo: Generating models for an existing DB -->
< p class = "page-source" >
Page source: < a href = "https://github.com/LispCookbook/cl-cookbook/blob/master/databases.md" > databases.md< / a >
< / p >
< / div >
< script type = "text/javascript" >
// Don't write the TOC on the index.
if (window.location.pathname != "/cl-cookbook/") {
$("#toc").toc({
content: "#content", // will ignore the first h1 with the site+page title.
headings: "h1,h2,h3,h4"});
}
$("#two-cols + ul").css({
"column-count": "2",
});
$("#contributors + ul").css({
"column-count": "4",
});
< / script >
< div >
< footer class = "footer" >
< hr / >
© 2002– 2021 the Common Lisp Cookbook Project
< / footer >
< / div >
< div id = "toc-btn" > T< br > O< br > C< / div >
< / div >
< script text = "javascript" >
HighlightLisp.highlight_auto({className: null});
< / script >
< script type = "text/javascript" >
function duckSearch() {
var searchField = document.getElementById("searchField");
if (searchField & & searchField.value) {
var query = escape("site:lispcookbook.github.io/cl-cookbook/ " + searchField.value);
window.location.href = "https://duckduckgo.com/?kj=b2& kf=-1& ko=1& q=" + query;
// https://duckduckgo.com/params
// kj=b2: blue header in results page
// kf=-1: no favicons
}
}
< / script >
< script async defer data-domain = "lispcookbook.github.io/cl-cookbook" src = "https://plausible.io/js/plausible.js" > < / script >
< / body >
< / html >
