emacs.d/clones/lisp/gigamonkeys.com/book/practical-web-programming-with-allegroserve.html

700 lines
51 KiB
HTML
Raw Normal View History

2022-08-02 12:34:59 +02:00
<HTML><HEAD><TITLE>Practical: Web Programming with AllegroServe</TITLE><LINK REL="stylesheet" TYPE="text/css" HREF="style.css"/></HEAD><BODY><DIV CLASS="copyright">Copyright &copy; 2003-2005, Peter Seibel</DIV><H1>26. Practical: Web Programming with AllegroServe</H1><P>In this chapter you'll look at one way to develop Web-based programs
in Common Lisp, using the open-source AllegroServe Web server. This
isn't meant as a full introduction to AllegroServe. And I'm certainly
not going to cover anything more than a tiny corner of the larger
topic of Web programming. My goal here is to cover enough of the
basics of using AllegroServe that you'll be able, in Chapter 29, to
develop an application for browsing a library of MP3 files and
streaming them to an MP3 client. Similarly, this chapter will serve as
a brief introduction to Web programming for folks new to the topic.</P><A NAME="a-30-second-intro-to-server-side-web-programming"><H2>A 30-Second Intro to Server-Side Web Programming</H2></A><P>While Web programming today typically involves quite a number of
software frameworks and different protocols, the core bits of Web
programming haven't changed much since they were invented in the
early 1990s. For simple applications, such as the one you'll write in
Chapter 29, you need to understand only a few key concepts, so I'll
review them quickly here. Experienced Web programmers can skim or
skip the rest of this section.<SUP>1</SUP></P><P>To start, you need to understand the roles the Web browser and the
Web server play in Web programming. While a modern browser comes with
a lot of bells and whistles, the core functionality of a Web browser
is to request Web pages from a Web server and then render them.
Typically those pages will be written in the Hypertext Markup
Language (HTML), which tells the browser how to render the page,
including where to insert inline images and links to other Web pages.
HTML consists of text <I>marked up</I> with <I>tags</I> that give the text
a structure that the browser uses when rendering the page. For
instance, a simple HTML document looks like this:</P><PRE>&lt;html&gt;
&lt;head&gt;
&lt;title&gt;Hello&lt;/title&gt;
&lt;/head&gt;
&lt;body&gt;
&lt;p&gt;Hello, world!&lt;/p&gt;
&lt;p&gt;This is a picture: &lt;img src=&quot;some-image.gif&quot;&gt;&lt;/p&gt;
&lt;p&gt;This is a &lt;a href=&quot;another-page.html&quot;&gt;link&lt;/a&gt; to another page.&lt;/p&gt;
&lt;/body&gt;
&lt;/html&gt;</PRE><P>Figure 26-1 shows how the browser renders this page.</P><P><IMG CLASS="figure" SRC="screenshots/new-sample.jpg"/></P><P><DIV CLASS="figure-caption">Figure 26-1. Sample Web page</DIV></P><P>The browser and server communicate using a protocol called the
Hypertext Transfer Protocol (HTTP). While you don't need to worry
about the details of the protocol, it's worth understanding that it
consists entirely of a sequence of requests initiated by the browser
and responses generated by the server. That is, the browser connects
to the Web server and sends a request that includes, at the least,
the desired URL and the version of HTTP that the browser speaks. The
browser can also include data in its request; that's how the browser
submits HTML forms to the server.</P><P>To reply to a request, the server sends a response made up of a set
of headers and a body. The headers contain information about the
body, such as what type of data it is (for instance, HTML, plain
text, or an image), and the body is the data itself, which is then
rendered by the browser. The server can also send an error response
telling the browser that its request couldn't be answered for some
reason.</P><P>And that's pretty much it. Once the browser has received the complete
response from the server, there's no communication between the
browser and the server until the next time the browser decides to
request a page from the server.<SUP>2</SUP> This is the main constraint of Web
programming--there's no way for code running on the server to affect
what the user sees in their browser unless the browser issues a new
request to the server.<SUP>3</SUP></P><P>Some Web pages, called <I>static</I> pages, are simply HTML files stored
on the Web server and served up when requested by the browser.
<I>Dynamic</I> pages, on the other hand, consist of HTML generated each
time the page is requested by a browser. For instance, a dynamic page
might be generated by querying a database and then constructing HTML
to represent the results of the query.<SUP>4</SUP></P><P>When generating its response to a request, server-side code has four
main pieces of information to act on. The first piece of information
is the requested URL. Typically, however, the URL is used by the Web
server itself to determine <I>what</I> code is responsible for
generating the response. Next, if the URL contains a question mark,
everything after the question mark is considered to be a <I>query
string</I>, which is typically ignored by the Web server except that it
makes it available to the code generating the response. Most of the
time the query string contains a set of key/value pairs. The request
from the browser can also contain <I>post data</I>, which also usually
consists of key/value pairs. Post data is typically used to submit
HTML forms. The key/value pairs supplied in either the query string
or the post data are collectively called the <I>query parameters</I>.</P><P>Finally, in order to string together a sequence of individual
requests from the same browser, code running in the server can <I>set
a cookie</I>, sending a special header in its response to the browser
that contains a bit of opaque data called a <I>cookie</I>. After a
cookie is set by a particular server, the browser will send the
cookie with each request it sends to that server. The browser doesn't
care about the data in the cookie--it just echoes it back to the
server for the server-side code to interpret however it wants.</P><P>These are the primitive elements on top of which 99 percent of
server-side Web programming is built. The browser sends a request,
the server finds some code to handle the request and runs it, and the
code uses query parameters and cookies to determine what to do.</P><A NAME="allegroserve"><H2>AllegroServe</H2></A><P>You can serve Web content using Common Lisp in a number of ways;
there are at least three open-source Web servers written in Common
Lisp as well as plug-ins such as
mod_lisp<SUP>5</SUP>
and Lisplets<SUP>6</SUP> that allow
the Apache Web server or any Java Servlet container to delegate
requests to a Lisp server running in a separate process.</P><P>For this chapter, you'll use a version of the open-source Web server
AllegroServe, originally written by John Foderaro at Franz Inc..
AllegroServe is included in the version of Allegro available from
Franz for use with this book. If you're not using Allegro, you can
use PortableAllegroServe, a friendly fork of the AllegroServe code
base, which includes an Allegro compatibility layer that allows
PortableAllegroServe to run on most Common Lisps. The code you'll
write in this chapter and in Chapter 29 should run in both vanilla
AllegroServe and PortableAllegroServe.</P><P>AllegroServe provides a programming model similar in spirit to Java
Servlets--each time a browser requests a page, AllegroServe parses
the request and looks up an object, called an <I>entity</I>, which
handles the request. Some entity classes provided as part of
AllegroServe know how to serve static content--either individual
files or the contents of a directory tree. Others, the ones I'll
spend most of this chapter discussing, run arbitrary Lisp code to
generate the response.<SUP>7</SUP></P><P>But before I get to that, you need to know how to start AllegroServe
and set it up to serve a few files. The first step is to load the
AllegroServe code into your Lisp image. In Allegro, you can simply
type <CODE>(require :aserve)</CODE>. In other Lisps (or in Allegro), you
can load PortableAllegroServe by loading the file <CODE>INSTALL.lisp</CODE>
at the top of the <CODE>portableaserve</CODE> directory tree. Loading
AllegroServe will create three new packages, <CODE>NET.ASERVE</CODE>,
<CODE>NET.HTML.GENERATOR</CODE>, and <CODE>NET.ASERVE.CLIENT</CODE>.<SUP>8</SUP></P><P>After loading the server, you start it with the function <CODE>start</CODE>
in the <CODE>NET.ASERVE</CODE> package. To have easy access to the symbols
exported from <CODE>NET.ASERVE</CODE>, from <CODE>COM.GIGAMONKEYS.HTML</CODE> (a
package I'll discuss in a moment), and from the rest of Common Lisp,
you should create a new package to play in like this:</P><PRE>CL-USER&gt; (defpackage :com.gigamonkeys.web
(:use :cl :net.aserve :com.gigamonkeys.html))
#&lt;The COM.GIGAMONKEYS.WEB package&gt;</PRE><P>Now switch to that package with this <CODE><B>IN-PACKAGE</B></CODE> expression:</P><PRE>CL-USER&gt; (in-package :com.gigamonkeys.web)
#&lt;The COM.GIGAMONKEYS.WEB package&gt;
WEB&gt; </PRE><P>Now you can use the exported names from <CODE>NET.ASERVE</CODE> without
qualification. The function <CODE>start</CODE> starts the server. It takes
quite a number of keyword parameters, but the only one you need to
pass is <CODE>:port</CODE>, which specifies the port to listen on. You
should probably use a high port such as 2001 instead of the default
port for HTTP servers, 80, because on Unix-derived operating systems
only the root user can listen on ports below 1024. To run
AllegroServe listening on port 80 on Unix, you'd need to start Lisp
as root and then use the <CODE>:setuid</CODE> and <CODE>:setgid</CODE> parameters
to tell <CODE>start</CODE> to switch its identity after opening the port.
You can start a server listening on port 2001 like this:</P><PRE>WEB&gt; (start :port 2001)
#&lt;WSERVER port 2001 @ #x72511c72&gt;</PRE><P>The server is now running in your Lisp. It's possible you'll get an
error that says something about &quot;port already in use&quot; when you try to
start the server. This means port 2001 is already in use by some
other server on your machine. In that case, the simplest fix is to
use a different port, supplying a different argument to <CODE>start</CODE>
and then using that value instead of 2001 in the URLs used throughout
this chapter.</P><P>You can continue to interact with Lisp via the REPL because
AllegroServe starts its own threads to handle requests from browsers.
This means, among other things, that you can use the REPL to get a
view into the guts of your server while it's running, which makes
debugging and testing a lot easier than if the server is a complete
black box.</P><P>Assuming you're running Lisp on the same machine as your browser, you
can check that the server is up and running by pointing your browser
at <CODE>http://localhost:2001/</CODE>. At this point you should get a
page-not-found error message in the browser since you haven't
published anything yet. But the error message will be from
AllegroServe; it'll say so at the bottom of the page. On the other
hand, if the browser displays an error dialog that says something
like &quot;The connection was refused when attempting to contact
localhost:2001,&quot; it means either that the server isn't running or
that you started it with a different port than 2001.</P><P>Now you can publish some files. Suppose you have a file
<CODE>hello.html</CODE> in the directory <CODE>/tmp/html</CODE> with the following
contents:</P><PRE>&lt;html&gt;
&lt;head&gt;
&lt;title&gt;Hello&lt;/title&gt;
&lt;/head&gt;
&lt;body&gt;
&lt;p&gt;Hello, world!&lt;/p&gt;
&lt;/body&gt;
&lt;/html&gt;</PRE><P>You can publish it individually with the <CODE>publish-file</CODE>
function.</P><PRE>WEB&gt; (publish-file :path &quot;/hello.html&quot; :file &quot;/tmp/html/hello.html&quot;)
#&lt;NET.ASERVE::FILE-ENTITY @ #x725eddea&gt;</PRE><P>The <CODE>:path</CODE> argument is the path that will appear in the URL
requested by the browser, while the <CODE>:file</CODE> argument is the name
of the file in the file system. After evaluating the
<CODE>publish-file</CODE> expression, you can point your browser to
<CODE>http://localhost:2001/hello.html</CODE>, and it should display a page
something like Figure 26-2.</P><P><IMG CLASS="figure" SRC="screenshots/hello-world.jpg"/></P><P><DIV CLASS="figure-caption">Figure 26-2. <CODE>http://localhost:2001/hello.html</CODE></DIV></P><P>You could also publish a whole directory tree of files using the
<CODE>publish-directory</CODE> function. First let's clear out the already
published entity with the following call to <CODE>publish-file</CODE>:</P><PRE>WEB&gt; (publish-file :path &quot;/hello.html&quot; :remove t)
NIL</PRE><P>Now you can publish the whole <CODE>/tmp/html/</CODE> directory (and all
its subdirectories) with the <CODE>publish-directory</CODE> function.</P><PRE>WEB&gt; (publish-directory :prefix &quot;/&quot; :destination &quot;/tmp/html/&quot;)
#&lt;NET.ASERVE::DIRECTORY-ENTITY @ #x72625aa2&gt;</PRE><P>In this case, the <CODE>:prefix</CODE> argument specifies the beginning of
the path part of URLs that should be handled by this entity. Thus, if
the server receives a request for
<CODE>http://localhost:2001/foo/bar.html</CODE>, the path is
<CODE>/foo/bar.html</CODE>, which starts with <CODE>/</CODE>. This path is then
translated to a filename by replacing the prefix, <CODE>/</CODE>, with the
destination, <CODE>/tmp/html/</CODE>. Thus, the URL
<CODE>http://localhost:2001/hello.html</CODE> will still be translated into
a request for the file <CODE>/tmp/html/hello.html</CODE>.</P><A NAME="generating-dynamic-content-with-allegroserve"><H2>Generating Dynamic Content with AllegroServe</H2></A><P>Publishing entities that generate dynamic content is nearly as simple
as publishing static content. The functions <CODE>publish</CODE> and
<CODE>publish-prefix</CODE> are the dynamic analogs of <CODE>publish-file</CODE>
and <CODE>publish-directory</CODE>. The basic idea of these two functions is
that you publish a function that will be called to generate the
response to a request for either a specific URL or any URL with a
given prefix. The function will be called with two arguments: an
object representing the request and the published entity. Most of
time you don't need to do anything with the entity object except to
pass it along to a couple macros I'll discuss in a moment. On the
other hand, you'll use the request object to obtain information
submitted by the browser--query parameters included in the URL or
data posted using an HTML form.</P><P>For a trivial example of using a function to generate dynamic
content, let's write a function that generates a page with a
different random number each time it's requested.</P><PRE>(defun random-number (request entity)
(with-http-response (request entity :content-type &quot;text/html&quot;)
(with-http-body (request entity)
(format
(request-reply-stream request)
&quot;&lt;html&gt;~@
&lt;head&gt;&lt;title&gt;Random&lt;/title&gt;&lt;/head&gt;~@
&lt;body&gt;~@
&lt;p&gt;Random number: ~d&lt;/p&gt;~@
&lt;/body&gt;~@
&lt;/html&gt;~@
&quot;
(random 1000)))))</PRE><P>The macros <CODE>with-http-response</CODE> and <CODE>with-http-body</CODE> are
part of AllegroServe. The former starts the process of generating an
HTTP response and can be used, as here, to specify things such as the
type of content that will be returned. It also handles various parts
of HTTP such as dealing with If-Modified-Since requests. The
<CODE>with-http-body</CODE> actually sends the HTTP response headers and
then executes its body, which should contain code that generates the
content of the reply. Within <CODE>with-http-response</CODE> but before the
<CODE>with-http-body</CODE>, you can add or change HTTP headers to be sent
in the reply. The function <CODE>request-reply-stream</CODE> is also part
of AllegroServe and returns the stream to which you should write
output intended to be sent to the browser.</P><P>As this function shows, you can just use <CODE><B>FORMAT</B></CODE> to print HTML to
the stream returned by <CODE>request-reply-stream</CODE>. In the next
section, I'll show you more convenient ways to programmatically
generate HTML.<SUP>9</SUP></P><P>Now you're ready to publish this function.</P><PRE>WEB&gt; (publish :path &quot;/random-number&quot; :function 'random-number)
#&lt;COMPUTED-ENTITY @ #x7262bab2&gt;</PRE><P>As it does in the <CODE>publish-file</CODE> function, the <CODE>:path</CODE>
argument specifies the path part of the URL that will result in this
function being invoked. The <CODE>:function</CODE> argument specifies
either the name or an actual function object. Using the name of a
function, as shown here, allows you to redefine the function later
without republishing and have AllegroServe use the new function
definition. After evaluating the call to <CODE>publish</CODE>, you can
point your browser at <CODE>http:// localhost:2001/random-number</CODE> to
get a page with a random number on it, as shown in Figure 26-3.</P><P><IMG CLASS="figure" SRC="screenshots/random-number.jpg"/></P><P><DIV CLASS="figure-caption">Figure 26-3. <CODE>http://localhost:2001/random-number</CODE></DIV></P><A NAME="generating-html"><H2>Generating HTML</H2></A><P>Although using <CODE><B>FORMAT</B></CODE> to emit HTML works fine for the simple
pages I've discussed so far, as you start building more elaborate
pages it'd be nice to have a more concise way to generate HTML.
Several libraries are available for generating HTML from an
s-expression representation including one, htmlgen, that's included
with AllegroServe. In this chapter you'll use a library called
FOO,<SUP>10</SUP> which is loosely modeled on Franz's htmlgen and whose
implementation you'll look at in more detail in Chapters 30 and 31.
For now, however, you just need to know how to use FOO.</P><P>Generating HTML from within Lisp is quite natural since s-expressions
and HTML are essentially isomorphic. You can represent HTML elements
with s-expressions by treating each element in HTML as a list
&quot;tagged&quot; with an appropriate first element, such as a keyword symbol
of the same name as the HTML tag. Thus, the HTML <CODE>&lt;p&gt;foo&lt;/p&gt;</CODE> is
represented by the s-expression <CODE>(:p &quot;foo&quot;)</CODE>. Because HTML
elements nest the same way lists in s-expressions do, this scheme
extends to more complex HTML. For instance, this HTML:</P><PRE>&lt;html&gt;
&lt;head&gt;
&lt;title&gt;Hello&lt;/title&gt;
&lt;/head&gt;
&lt;body&gt;
&lt;p&gt;Hello, world!&lt;/p&gt;
&lt;/body&gt;
&lt;/html&gt;</PRE><P>could be represented with the following s-expression:</P><PRE>(:html
(:head (:title &quot;Hello&quot;))
(:body (:p &quot;Hello, world!&quot;)))</PRE><P>HTML elements with attributes complicate things a bit but not in an
insurmountable way. FOO supports two ways of including attributes in
a tag. One is to simply follow the first item of the list with
keyword/value pairs. The first element that follows a keyword/value
pair that's not itself a keyword symbol marks the beginning of the
element's contents. Thus, you'd represent this HTML:</P><PRE>&lt;a href=&quot;foo.html&quot;&gt;This is a link&lt;/a&gt;</PRE><P>with the following s-expression:</P><PRE>(:a :href &quot;foo.html&quot; &quot;This is a link&quot;)</PRE><P>The other syntax FOO supports is to group the tag name and attributes
into their own list like this:</P><PRE>((:a :href &quot;foo.html&quot;) &quot;This is link.&quot;)</PRE><P>FOO can use the s-expression representation of HTML in two ways. The
function <CODE>emit-html</CODE> takes an HTML s-expression and outputs the
corresponding HTML.</P><PRE>WEB&gt; (emit-html '(:html (:head (:title &quot;Hello&quot;)) (:body (:p &quot;Hello, world!&quot;))))
&lt;html&gt;
&lt;head&gt;
&lt;title&gt;Hello&lt;/title&gt;
&lt;/head&gt;
&lt;body&gt;
&lt;p&gt;Hello, world!&lt;/p&gt;
&lt;/body&gt;
&lt;/html&gt;
T</PRE><P>However, <CODE>emit-html</CODE> isn't always the most efficient way to
generate HTML because its argument must be a complete s-expression
representation of the HTML to be generated. While it's easy to build
such a representation, it's not always particularly efficient. For
instance, suppose you wanted to make an HTML page containing a list
of 10,000 random numbers. You could build the s-expression using a
backquote template and then pass it to <CODE>emit-html</CODE> like this:</P><PRE>(emit-html
`(:html
(:head
(:title &quot;Random numbers&quot;))
(:body
(:h1 &quot;Random numbers&quot;)
(:p ,@(loop repeat 10000 collect (random 1000) collect &quot; &quot;)))))</PRE><P>However, this has to build a tree containing a 10,000-element list
before it can even start emitting HTML, and the whole s-expression
will become garbage as soon as the HTML is emitted. To avoid this
inefficiency, FOO also provides a macro <CODE>html</CODE>, which allows you
to embed bits of Lisp code in the middle of an HTML s-expression.</P><P>Literal values such as strings and numbers in the input to
<CODE>html</CODE> are interpolated into the output HTML. Likewise, symbols
are treated as variable references, and code is generated to emit
their value at runtime. Thus, both of these:</P><PRE>(html (:p &quot;foo&quot;))
(let ((x &quot;foo&quot;)) (html (:p x)))</PRE><P>will emit the following:</P><PRE>&lt;p&gt;foo&lt;/p&gt;</PRE><P>List forms that don't start with a keyword symbol are assumed to be
code and are embedded in the generated code. Any values the embedded
code returns will be ignored, but the code can emit more HTML by
calling <CODE>html</CODE> itself. For instance, to emit the contents of a
list in HTML, you might write this:</P><PRE>(html (:ul (dolist (item (list 1 2 3)) (html (:li item)))))</PRE><P>which will emit the following HTML:</P><PRE>&lt;ul&gt;
&lt;li&gt;1&lt;/li&gt;
&lt;li&gt;2&lt;/li&gt;
&lt;li&gt;3&lt;/li&gt;
&lt;/ul&gt;</PRE><P>If you want to emit the value of a list form, you must wrap it in the
pseudotag <CODE>:print</CODE>. Thus, this expression:</P><PRE>(html (:p (+ 1 2)))</PRE><P>generates this HTML after computing and discarding the value
<CODE>3</CODE>:</P><PRE>&lt;p&gt;&lt;/p&gt;</PRE><P>To emit the <CODE>3</CODE>, you must write this:</P><PRE>(html (:p (:print (+ 1 2))))</PRE><P>Or you could compute the value and store it in a variable outside the
call to <CODE>html</CODE> like this:</P><PRE>(let ((x (+ 1 2))) (html (:p x)))</PRE><P>Thus, you can use the <CODE>html</CODE> macro to generate the list of
random numbers like this:</P><PRE>(html
(:html
(:head
(:title &quot;Random numbers&quot;))
(:body
(:h1 &quot;Random numbers&quot;)
(:p (loop repeat 10 do (html (:print (random 1000)) &quot; &quot;))))))</PRE><P>The macro version will be quite a bit more efficient than the
<CODE>emit-html</CODE> version. Not only do you never have to generate an
s-expression representing the whole page, also much of the work that
<CODE>emit-html</CODE> does at runtime to interpret the s-expression will be
done once, when the macro is expanded, rather than every time the code
is run.</P><P>You can control where the output generated by both <CODE>html</CODE> and
<CODE>emit-html</CODE> is sent with the macro <CODE>with-html-output</CODE>,
which is part of the FOO library. Thus, you can use the
<CODE>with-html-output</CODE> and <CODE>html</CODE> macros from FOO to rewrite
<CODE>random-number</CODE> like this:</P><PRE>(defun random-number (request entity)
(with-http-response (request entity :content-type &quot;text/html&quot;)
(with-http-body (request entity)
(with-html-output ((request-reply-stream request))
(html
(:html
(:head (:title &quot;Random&quot;))
(:body
(:p &quot;Random number: &quot; (:print (random 1000))))))))))</PRE><A NAME="html-macros"><H2>HTML Macros</H2></A><P>Another feature of FOO is that it allows you to define HTML &quot;macros&quot;
that can translate arbitrary forms into HTML s-expressions that the
<CODE>html</CODE> macro understands. For instance, suppose you frequently
find yourself writing pages of this form:</P><PRE>(:html
(:head (:title &quot;Some title&quot;))
(:body
(:h1 &quot;Some title&quot;)
<I>... stuff ...</I>))</PRE><P>You could define an HTML macro to capture that pattern like this:</P><PRE>(define-html-macro :standard-page ((&amp;key title) &amp;body body)
`(:html
(:head (:title ,title))
(:body
(:h1 ,title)
,@body)))</PRE><P>Now you can use the &quot;tag&quot; <CODE>:standard-page</CODE> in your s-expression
HTML, and it'll be expanded before being interpreted or compiled. For
instance, the following:</P><PRE>(html (:standard-page (:title &quot;Hello&quot;) (:p &quot;Hello, world.&quot;)))</PRE><P>generates the following HTML:</P><PRE>&lt;html&gt;
&lt;head&gt;
&lt;title&gt;Hello&lt;/title&gt;
&lt;/head&gt;
&lt;body&gt;
&lt;h1&gt;Hello&lt;/h1&gt;
&lt;p&gt;Hello, world.&lt;/p&gt;
&lt;/body&gt;
&lt;/html&gt;</PRE><A NAME="query-parameters"><H2>Query Parameters</H2></A><P>Of course, generating HTML output is only half of Web programming.
The other thing you need to do is get input from the user. As I
discussed in the &quot;A 30-Second Intro to Server-Side Web Programming&quot;
section, when a browser requests a page from a Web server, it can
send query parameters in the URL and post data, both of which act as
input to the server-side code.</P><P>AllegroServe, like most Web programming frameworks, takes care of
parsing both these sources of input for you. By the time your
published functions are called, all the key/value pairs from the
query string and/or post data have been decoded and placed into an
alist that you can retrieve from the request object with the function
<CODE>request-query</CODE>. The following function returns a page showing
all the query parameters it receives:</P><PRE>(defun show-query-params (request entity)
(with-http-response (request entity :content-type &quot;text/html&quot;)
(with-http-body (request entity)
(with-html-output ((request-reply-stream request))
(html
(:standard-page
(:title &quot;Query Parameters&quot;)
(if (request-query request)
(html
(:table :border 1
(loop for (k . v) in (request-query request)
do (html (:tr (:td k) (:td v))))))
(html (:p &quot;No query parameters.&quot;)))))))))
(publish :path &quot;/show-query-params&quot; :function 'show-query-params)</PRE><P>If you give your browser a URL with a query string in it like the
following:</P><PRE>http://localhost:2001/show-query-params?foo=bar&amp;baz=10</PRE><P>you should get back a page similar to the one shown in Figure 26-4.</P><P><IMG CLASS="figure" SRC="screenshots/show-query-params-1a.jpg"/></P><P><DIV CLASS="figure-caption">Figure 26-4. <CODE>http://localhost:2001/show-query-params?foo=bar&amp;baz=10</CODE></DIV></P><P>To generate some post data, you need an HTML form. The following
function generates a simple form, which submits its data to
<CODE>show-query-params</CODE>:</P><PRE>(defun simple-form (request entity)
(with-http-response (request entity :content-type &quot;text/html&quot;)
(with-http-body (request entity)
(let ((*html-output* (request-reply-stream request)))
(html
(:html
(:head (:title &quot;Simple Form&quot;))
(:body
(:form :method &quot;POST&quot; :action &quot;/show-query-params&quot;
(:table
(:tr (:td &quot;Foo&quot;)
(:td (:input :name &quot;foo&quot; :size 20)))
(:tr (:td &quot;Password&quot;)
(:td (:input :name &quot;password&quot; :type &quot;password&quot; :size 20))))
(:p (:input :name &quot;submit&quot; :type &quot;submit&quot; :value &quot;Okay&quot;)
(:input ::type &quot;reset&quot; :value &quot;Reset&quot;))))))))))
(publish :path &quot;/simple-form&quot; :function 'simple-form)</PRE><P>Point your browser to <CODE>http://localhost:2001/simple-form</CODE>, and
you should see a page like the one in Figure 26-5.</P><P>If you fill in the form with the &quot;abc&quot; and &quot;def&quot; values, clicking the
Okay button should take you to a page like the one in Figure 26-6.</P><P><IMG CLASS="figure" SRC="screenshots/simple-form.jpg"/></P><P><DIV CLASS="figure-caption">Figure 26-5. <CODE>http://localhost:2001/simple-form</CODE></DIV></P><P><IMG CLASS="figure" SRC="screenshots/show-query-params-2.jpg"/></P><P><DIV CLASS="figure-caption">Figure 26-6. Result of submitting the simple form</DIV></P><P>However, most of the time you won't need to iterate over all the query
parameters; you'll want to pick out individual parameters. For
instance, you might want to modify <CODE>random-number</CODE> so the limit
value you pass to <CODE><B>RANDOM</B></CODE> can be supplied via a query parameter.
In that case, you use the function <CODE>request-query-value</CODE>, which
takes the request object and the name of the parameter whose value
you want and returns the value as a string or <CODE><B>NIL</B></CODE> if no such
parameter has been supplied. A parameterizable version of
<CODE>random-number</CODE> might look like this:</P><PRE>(defun random-number (request entity)
(with-http-response (request entity :content-type &quot;text/html&quot;)
(with-http-body (request entity)
(let* ((*html-output* (request-reply-stream request))
(limit-string (or (request-query-value &quot;limit&quot; request) &quot;&quot;))
(limit (or (parse-integer limit-string :junk-allowed t) 1000)))
(html
(:html
(:head (:title &quot;Random&quot;))
(:body
(:p &quot;Random number: &quot; (:print (random limit))))))))))</PRE><P>Because <CODE>request-query-value</CODE> can return either <CODE><B>NIL</B></CODE> or an
empty string, you have to deal with both those cases when parsing the
parameter into a number to pass to <CODE><B>RANDOM</B></CODE>. You can deal with a
<CODE><B>NIL</B></CODE> value when you bind <CODE>limit-string</CODE>, binding it to
<CODE>&quot;&quot;</CODE> if there's no &quot;limit&quot; query parameter. Then you can use the
<CODE>:junk-allowed</CODE> argument to <CODE><B>PARSE-INTEGER</B></CODE> to ensure that it
returns either <CODE><B>NIL</B></CODE> (if it can't parse an integer from the string
given) or an integer. In the section &quot;A Small Application Framework,&quot;
you'll develop some macros to make it easier to deal with grabbing
query parameters and converting them to various types.</P><A NAME="cookies"><H2>Cookies</H2></A><P>In AllegroServe you can send a Set-Cookie header that tells the
browser to save a cookie and send it along with subsequent requests
by calling the function <CODE>set-cookie-header</CODE> within the body of
<CODE>with-http-response</CODE> but before the call to
<CODE>with-http-body</CODE>. The first argument to the function is the
request object, and the remaining arguments are keyword arguments
used to set the various properties of the cookie. The only two you
must pass are the <CODE>:name</CODE> and <CODE>:value</CODE> arguments, both of
which should be strings. The other possible arguments that affect the
cookie sent to the browser are <CODE>:expires</CODE>, <CODE>:path</CODE>,
<CODE>:domain</CODE>, and <CODE>:secure</CODE>.</P><P>Of these, you need to worry only about <CODE>:expires</CODE>. It controls
how long the browser should save the cookie. If <CODE>:expires</CODE> is
<CODE><B>NIL</B></CODE> (the default), the browser will save the cookie only until
it exits. Other possible values are <CODE>:never</CODE>, which means the
cookie should be kept forever, or a universal time as returned by
<CODE><B>GET-UNIVERSAL-TIME</B></CODE> or <CODE><B>ENCODE-UNIVERSAL-TIME</B></CODE>. An
<CODE>:expires</CODE> of zero tells the client to immediately discard an
existing cookie.<SUP>11</SUP></P><P>After you've set a cookie, you can use the function
<CODE>get-cookie-values</CODE> to get an alist containing one name/value
pair for each cookie sent by the browser. From that alist, you can
pick out individual cookie values using <CODE><B>ASSOC</B></CODE> and <CODE><B>CDR</B></CODE>.</P><P>The following function shows the names and values of all the cookies
sent by the browser:</P><PRE>(defun show-cookies (request entity)
(with-http-response (request entity :content-type &quot;text/html&quot;)
(with-http-body (request entity)
(with-html-output ((request-reply-stream request))
(html
(:standard-page
(:title &quot;Cookies&quot;)
(if (null (get-cookie-values request))
(html (:p &quot;No cookies.&quot;))
(html
(:table
(loop for (key . value) in (get-cookie-values request)
do (html (:tr (:td key) (:td value)))))))))))))
(publish :path &quot;/show-cookies&quot; :function 'show-cookies)</PRE><P>The first time you load the page
<CODE>http://localhost:2001/show-cookies</CODE> it should say &quot;No cookies&quot;
as shown in Figure 26-7 since you haven't set any yet.</P><P><IMG CLASS="figure" SRC="screenshots/show-cookies-no-cookies.jpg"/></P><P><DIV CLASS="figure-caption">Figure 26-7. <CODE>http://localhost:2001/show-cookies</CODE> with no cookies</DIV></P><P>To set a cookie, you need another function, such as the following:</P><PRE>(defun set-cookie (request entity)
(with-http-response (request entity :content-type &quot;text/html&quot;)
(set-cookie-header request :name &quot;MyCookie&quot; :value &quot;A cookie value&quot;)
(with-http-body (request entity)
(with-html-output ((request-reply-stream request))
(html
(:standard-page
(:title &quot;Set Cookie&quot;)
(:p &quot;Cookie set.&quot;)
(:p (:a :href &quot;/show-cookies&quot; &quot;Look at cookie jar.&quot;))))))))
(publish :path &quot;/set-cookie&quot; :function 'set-cookie)</PRE><P>If you enter the URL <CODE>http://localhost:2001/set-cookie</CODE>, your
browser should display a page like the one in Figure 26-8.
Additionally, the server will send a Set-Cookie header with a cookie
named &quot;MyCookie&quot; with &quot;A cookie value&quot; as its value. If you click the
link <I>Look at cookie jar</I>, you'll be taken to the
<CODE>/show-cookies</CODE> page where you'll see the new cookie, as shown
in Figure 26-9. Because you didn't specify an <CODE>:expires</CODE>
argument, the browser will continue to send the cookie with each
request until you quit the browser.</P><P><IMG CLASS="figure" SRC="screenshots/set-cookie.jpg"/></P><P><DIV CLASS="figure-caption">Figure 26-8. <CODE>http://localhost:2001/set-cookie</CODE></DIV></P><P><IMG CLASS="figure" SRC="screenshots/show-cookies-one-cookie.jpg"/></P><P><DIV CLASS="figure-caption">Figure 26-9. <CODE>http://localhost:2001/show-cookies</CODE> after setting a cookie</DIV></P><A NAME="a-small-application-framework"><H2>A Small Application Framework</H2></A><P>Although AllegroServe provides fairly straightforward access to all
the basic facilities you need to write server-side Web code (access
to query parameters from both the URL's query string and the post
data; the ability to set cookies and retrieve their values; and, of
course, the ability to generate the response sent back to the
browser), there's a fair bit of annoyingly repetitive code.</P><P>For instance, every HTML-generating function you write is going to
take the arguments <CODE>request</CODE> and <CODE>entity</CODE> and then will
contain calls to <CODE>with-http-response</CODE>,
<CODE>with-http-response</CODE>, and--if you're going to use FOO to
generate HTML--<CODE>with-html-output</CODE>. Then, in functions that need
to get at query parameters, there will be a bunch of calls to
<CODE>request-query-value</CODE> and then more code to convert the string
returned to whatever type you actually want. Finally, you need to
remember to <CODE>publish</CODE> the function.</P><P>To reduce the amount of boilerplate you have to write, you can write
a small framework on top of AllegroServe to make it easier to define
functions that handle requests for a particular URL.</P><P>The basic approach will be to define a macro,
<CODE>define-url-function</CODE>, that you'll use to define functions that
will automatically be published via <CODE>publish</CODE>. This macro will
expand into a <CODE><B>DEFUN</B></CODE> that contains the appropriate boilerplate as
well as code to publish the function under a URL of the same name.
It'll also take care of generating code to extract values from query
parameters and cookies and to bind them to variables declared in the
function's parameter list. Thus, the basic form of a
<CODE>define-url-function</CODE> definition is this:</P><PRE>(define-url-function <I>name</I> (<I>request</I> <I>query-parameter</I>*)
<I>body</I>)</PRE><P>where the <I>body</I> is the code to emit the HTML of the page. It'll be
wrapped in a call to FOO's <CODE>html</CODE> macro, so for simple pages it
might contain nothing but s-expression HTML.</P><P>Within the body, the query parameter variables will be bound to
values of query parameters with the same name or from a cookie. In
the simplest case, a query parameter's value will be the string taken
from the query parameter or post data field of the same name. If the
query parameter is specified with a list, you can also specify an
automatic type conversion, a default value, and whether to look for
and save the value of the parameter in a cookie. The complete syntax
for a <I>query-parameter</I> is as follows:</P><PRE><I>name</I> | (<I>name</I> <I>type</I> [<I>default-value</I>] [<I>stickiness</I>])</PRE><P>The <I>type</I> must be a name recognized by <CODE>define-url-function</CODE>.
I'll discuss in a moment how to define new types. The
<I>default-value</I> must be a value of the given type. Finally,
<I>stickiness</I>, if supplied, indicates that the parameter's value
should be taken from an appropriately named cookie if no query
parameter is supplied and that a Set-Cookie header should be sent in
the response that saves the value in the cookie of the same name.
Thus, a sticky parameter, after being explicitly supplied a value via
a query parameter, will keep that value on subsequent requests of the
page even when no query parameter is supplied.</P><P>The name of the cookie used depends on the value of <I>stickiness</I>:
with a value of <CODE>:global</CODE>, the cookie will be named the same as
the parameter. Thus, different functions that use globally sticky
parameters with the same name will share the value. If <I>stickiness</I>
is <CODE>:package</CODE>, then the cookie name is constructed from the name
of the parameter and the package of the function's name; this allows
functions in the same package to share values but not have to worry
about stomping on parameters of functions in other packages. Finally,
a parameter with a <I>stickiness</I> value of <CODE>:local</CODE> will use a
cookie made from the name of the parameter, the package of the
function name, and the function name, making it unique to that
function.</P><P>For instance, you can use <CODE>define-url-function</CODE> to replace the
previous eleven-line definition of <CODE>random-page</CODE> with this
five-line version:</P><PRE>(define-url-function random-number (request (limit integer 1000))
(:html
(:head (:title &quot;Random&quot;))
(:body
(:p &quot;Random number: &quot; (:print (random limit))))))</PRE><P>If you wanted the limit argument to be sticky, you could change the
limit declaration to <CODE>(limit integer 1000 :local)</CODE>.</P><A NAME="the-implementation"><H2>The Implementation</H2></A><P>I'll explain the implementation of <CODE>define-url-function</CODE> from the
top down. The macro itself looks like this:</P><PRE>(defmacro define-url-function (name (request &amp;rest params) &amp;body body)
(with-gensyms (entity)
(let ((params (mapcar #'normalize-param params)))
`(progn
(defun ,name (,request ,entity)
(with-http-response (,request ,entity :content-type &quot;text/html&quot;)
(let* (,@(param-bindings name request params))
,@(set-cookies-code name request params)
(with-http-body (,request ,entity)
(with-html-output ((request-reply-stream ,request))
(html ,@body))))))
(publish :path ,(format nil &quot;/~(~a~)&quot; name) :function ',name)))))</PRE><P>Let's take it bit by bit, starting with the first few lines.</P><PRE>(defmacro define-url-function (name (request &amp;rest params) &amp;body body)
(with-gensyms (entity)
(let ((params (mapcar #'normalize-param params)))</PRE><P>Up to here you're just getting ready to generate code. You
<CODE><B>GENSYM</B></CODE> a symbol to use later as the name of the entity parameter
in the <CODE><B>DEFUN</B></CODE>. Then you normalize the parameters, converting
plain symbols to list form using this function:</P><PRE>(defun normalize-param (param)
(etypecase param
(list param)
(symbol `(,param string nil nil))))</PRE><P>In other words, declaring a parameter with just a symbol is the same
as declaring a nonsticky, string parameter with no default value.</P><P>Then comes the <CODE><B>PROGN</B></CODE>. You must expand into a <CODE><B>PROGN</B></CODE> because
you need to generate code to do two things: define a function with
<CODE><B>DEFUN</B></CODE> and call <CODE>publish</CODE>. You should define the function
first so if there's an error in the definition, the function won't be
published. The first two lines of the <CODE><B>DEFUN</B></CODE> are just
boilerplate.</P><PRE>(defun ,name (,request ,entity)
(with-http-response (,request ,entity :content-type &quot;text/html&quot;) </PRE><P>Now you do the real work. The following two lines generate the
bindings for the parameters specified in <CODE>define-url-function</CODE>
other than <CODE>request</CODE> and the code that calls
<CODE>set-cookie-header</CODE> for the sticky parameters. Of course, the
real work is done by helper functions that you'll look at in a
moment.<SUP>12</SUP></P><PRE> (let* (,@(param-bindings name request params))
,@(set-cookies-code name request params)</PRE><P>The rest is just more boilerplate, putting the body from the
<CODE>define-url-function</CODE> definition in the appropriate context of
<CODE>with-http-body</CODE>, <CODE>with-html-output</CODE>, and <CODE>html</CODE>
macros. Then comes the call to <CODE>publish</CODE>.</P><PRE> (publish :path ,(format nil &quot;/~(~a~)&quot; name) :function ',name)</PRE><P>The expression <CODE>(format nil &quot;/~(~a~)&quot; name)</CODE> is evaluated at
macro expansion time, generating a string consisting of /, followed
by an all-lowercase version of the name of the function you're about
to define. That string becomes the <CODE>:path</CODE> argument to publish,
while the function name is interpolated as the <CODE>:function</CODE>
argument.</P><P>Now let's look at the helper functions used to generate the
<CODE><B>DEFUN</B></CODE> form. To generate parameter bindings, you need to loop
over the <CODE>params</CODE> and collect a snippet of code for each one,
generated by <CODE>param-binding</CODE>. That snippet will be a list
containing the name of the variable to bind and the code that will
compute the value of that variable. The exact form of code used to
compute the value will depend on the type of the parameter, whether
it's sticky, and the default value, if any. Because you already
normalized the params, you can use <CODE><B>DESTRUCTURING-BIND</B></CODE> to take
them apart in <CODE>param-binding</CODE>.</P><PRE>(defun param-bindings (function-name request params)
(loop for param in params
collect (param-binding function-name request param)))
(defun param-binding (function-name request param)
(destructuring-bind (name type &amp;optional default sticky) param
(let ((query-name (symbol-&gt;query-name name))
(cookie-name (symbol-&gt;cookie-name function-name name sticky)))
`(,name (or
(string-&gt;type ',type (request-query-value ,query-name ,request))
,@(if cookie-name
(list `(string-&gt;type ',type (get-cookie-value ,request ,cookie-name))))
,default)))))</PRE><P>The function <CODE>string-&gt;type</CODE>, which you use to convert strings
obtained from the query parameters and cookies to the desired type, is
a generic function with the following signature:</P><PRE>(defgeneric string-&gt;type (type value))</PRE><P>To make a particular name usable as a type name for a query
parameter, you just need to define a method on <CODE>string-&gt;type</CODE>.
You'll need to define at least a method specialized on the symbol
<CODE>string</CODE> since that's the default type. Of course, that's pretty
easy. Since browsers sometimes submit forms with empty strings to
indicate no value was supplied for a particular value, you'll want to
convert an empty string to <CODE><B>NIL</B></CODE> as this method does:</P><PRE>(defmethod string-&gt;type ((type (eql 'string)) value)
(and (plusp (length value)) value))</PRE><P>You can add conversions for other types needed by your application.
For instance, to make <CODE>integer</CODE> usable as a query parameter type
so you can handle the <CODE>limit</CODE> parameter of <CODE>random-page</CODE>,
you might define this method:</P><PRE>(defmethod string-&gt;type ((type (eql 'integer)) value)
(parse-integer (or value &quot;&quot;) :junk-allowed t))</PRE><P>Another helper function used in the code generated by
<CODE>param-binding</CODE> is <CODE>get-cookie-value</CODE>, which is just a bit
of sugar around the <CODE>get-cookie-values</CODE> function provided by
AllegroServe. It looks like this:</P><PRE>(defun get-cookie-value (request name)
(cdr (assoc name (get-cookie-values request) :test #'string=)))</PRE><P>The functions that compute the query parameter and cookies names are
similarly straightforward.</P><PRE>(defun symbol-&gt;query-name (sym)
(string-downcase sym))
(defun symbol-&gt;cookie-name (function-name sym sticky)
(let ((package-name (package-name (symbol-package function-name))))
(when sticky
(ecase sticky
(:global
(string-downcase sym))
(:package
(format nil &quot;~(~a:~a~)&quot; package-name sym))
(:local
(format nil &quot;~(~a:~a:~a~)&quot; package-name function-name sym))))))</PRE><P>To generate the code that sets cookies for sticky parameters, you
again loop over the list of parameters, this time collecting a
snippet of code for each sticky param. You can use the <CODE>when
</CODE>and <CODE>collect it</CODE> <CODE><B>LOOP </B></CODE>forms to collect only the
non-<CODE><B>NIL</B></CODE> values returned by <CODE>set-cookie-code</CODE>.</P><PRE>(defun set-cookies-code (function-name request params)
(loop for param in params
when (set-cookie-code function-name request param) collect it))
(defun set-cookie-code (function-name request param)
(destructuring-bind (name type &amp;optional default sticky) param
(declare (ignore type default))
(if sticky
`(when ,name
(set-cookie-header
,request
:name ,(symbol-&gt;cookie-name function-name name sticky)
:value (princ-to-string ,name))))))</PRE><P>One of the advantages of defining macros in terms of helper functions
like this is that it's easy to make sure the individual bits of code
you're generating look right. For instance, you can check that the
following <CODE>set-cookie-code</CODE>:</P><PRE>(set-cookie-code 'foo 'request '(x integer 20 :local))</PRE><P>generates something like this:</P><PRE>(WHEN X
(SET-COOKIE-HEADER REQUEST
:NAME &quot;com.gigamonkeys.web:foo:x&quot;
:VALUE (PRINC-TO-STRING X)))</PRE><P>Assuming this code will occur in a context where <CODE>x</CODE> is the name
of a variable, this looks good.</P><P>Once again, macros have allowed you to distill the code you need to
write down to its essence--in this case, the data you want to extract
from the request and the HTML you want to generate. That said, this
framework isn't meant to be the be-all and end-all of Web application
frameworks--it's just a little sugar to make it a bit easier to write
simple apps like the one you'll write in Chapter 29.</P><P>But before you can get to that, you need to write the guts of the
application for which the Chapter 29 application will be the user
interface. You'll start in the next chapter with a souped-up version
of the database you wrote in Chapter 3, this time to keep track of ID3
data extracted from MP3 files.</P><HR/><DIV CLASS="notes"><P><SUP>1</SUP>Readers new to Web programming
will probably need to supplement this introduction with a more
in-depth tutorial or two. You can find a good set of online tutorials
at <CODE>http://www.jmarshall.com/easy/</CODE>.</P><P><SUP>2</SUP>Loading a single Web page may
actually involve multiple requests--to render the HTML of a page
containing inline images, the browser must request each image
individually and then insert each into the appropriate place in the
rendered HTML.</P><P><SUP>3</SUP>Much of the complexity around Web
programming is a result of trying to work around this fundamental
limitation in order to provide a user experience that's more like the
interactivity provided by desktop applications.</P><P><SUP>4</SUP>Unfortunately,
<I>dynamic</I> is somewhat overloaded in the Web world. The phrase
<I>Dynamic HTML</I> refers to HTML containing embedded code, usually in
the language JavaScript, that can be executed in the browser without
further communication with the Web server. Used with some discretion,
Dynamic HTML can improve the usability of a Web-based application
since, even with high-speed Internet connections, making a request to
a Web server, receiving the response, and rendering the new page can
take a noticeable amount of time. To further confuse things,
dynamically generated pages (in other words, generated on the server)
could also contain Dynamic HTML (code to be run on the client.) For
the purposes of this book, you'll stick to dynamically generating
plain old nondynamic HTML.</P><P><SUP>5</SUP><CODE>http://www.fractalconcept.com/asp/html/mod_lisp.html</CODE></P><P><SUP>6</SUP><CODE>http://lisplets.sourceforge.net/</CODE></P><P><SUP>7</SUP>AllegroServe also provides a framework
called <I>Webactions</I> that's analogous to JSPs in the Java
world--instead of writing code that generates HTML, with Webactions
you write pages that are essentially HTML with a bit of magic foo
that turns into code to be run when the page is served. I won't cover
Webactions in this book.</P><P><SUP>8</SUP>Loading
PortableAllegroServe will create some other packages for the
compatibility libraries, but the packages you'll care about are those
three.</P><P><SUP>9</SUP>The <CODE>~@</CODE> followed by a newline tells
<CODE><B>FORMAT</B></CODE> to ignore whitespace after the newline, which allows you
to indent your code nicely without adding a bunch of whitespace to
the HTML. Since white-space is typically not significant in HTML,
this doesn't matter to the browser, but it makes the generated HTML
source look a bit nicer to humans.</P><P><SUP>10</SUP>FOO is a recursive tautological acronym for <I>FOO Outputs
Output</I>.</P><P><SUP>11</SUP>For information about the meaning of the other
parameters, see the AllegroServe documentation and RFC 2109, which
describes the cookie mechanism.</P><P><SUP>12</SUP>You need to use <CODE><B>LET*</B></CODE> rather than a <CODE><B>LET</B></CODE> to
allow the default value forms for parameters to refer to parameters
that appear earlier in the parameter list. For example, you could
write this:</P><PRE>(define-url-function (request (x integer 10) (y integer (* 2 x))) ...)</PRE><P>and the value of <CODE>y</CODE>, if not explicitly supplied, would be twice
the value of <CODE>x</CODE>.</P></DIV></BODY></HTML>