# page_content

c2: The SQLite Amalgamation
c3: 4. Download Copies Of The Precompiled Amalgamation
c5: amalgamation.html#download_copies_of_the_precompiled_amalgamation
c4:
The amalgamation and
the sqlite3.h header file are available on
the download page as a file
named sqlite-amalgamation-X.zip
where the X is replaced by the appropriate version number.


c2: The SQLite Amalgamation
c3: 5. Building The Amalgamation From Canonical Source Code
c5: amalgamation.html#building_the_amalgamation_from_canonical_source_code
c4:
To build the amalgamation (either the full amalgamation or the
split amalgamation), first
get the canonical source code from one of the three servers.
Then, on both unix-like systems and on Windows systems that have the
free MinGW development environment
installed, the amalgamation can be built using the
following commands:

sh configure
make sqlite3.c


To build using Microsoft Visual C++, run this command:

nmake /f makefile.msc sqlite3.c


In both cases, the split amalgamation can be obtained by
substituting "sqlite3-all.c" for "sqlite3.c" as the make target.


c2: The SQLite Amalgamation
c3: 5.1. Dependencies
c5: amalgamation.html#dependencies
c4:
The build process makes extensive use of the
Tcl scripting language.  You will need to have a
copy of TCL installed in order for the make targets above to work.
Easy-to-use installers can be obtained from http://www.tcl-lang.org/.
Many unix workstations have Tcl installed by default.


c2: SQLite FTS3 and FTS4 Extensions
c3: Overview
c5: fts3.html#overview
c4:
FTS3 and FTS4 are SQLite virtual table modules that allows users to perform
  full-text searches on a set of documents. The most common (and effective)
  way to describe full-text searches is "what Google, Yahoo, and Bing do
  with documents placed on the World Wide Web". Users input a term, or series
  of terms, perhaps connected by a binary operator or grouped together into a
  phrase, and the full-text query system finds the set of documents that best
  matches those terms considering the operators and groupings the user has
  specified. This article describes the deployment and usage of FTS3 and FTS4.


  FTS1 and FTS2 are obsolete full-text search modules for SQLite.  There are known
  issues with these older modules and their use should be avoided.
  Portions of the original FTS3 code were contributed to the SQLite project
  by Scott Hess of Google. It is now
  developed and maintained as part of SQLite.


c2: SQLite FTS3 and FTS4 Extensions
c3: 1. Introduction to FTS3 and FTS4
c5: fts3.html#introduction_to_fts3_and_fts4
c4:
The FTS3 and FTS4 extension modules allows users to create special tables with a
  built-in full-text index (hereafter "FTS tables"). The full-text index
  allows the user to efficiently query the database for all rows that contain
  one or more words (hereafter "tokens"), even if the table
  contains many large documents.


  For example, if each of the 517430 documents in the
  "Enron E-Mail Dataset"
  is inserted into both an FTS table and an ordinary SQLite table
  created using the following SQL script:

CREATE VIRTUAL TABLE enrondata1 USING fts3(content TEXT);     /* FTS3 table */
CREATE TABLE enrondata2(content TEXT);                        /* Ordinary table */



  Then either of the two queries below may be executed to find the number of
  documents in the database that contain the word "linux" (351). Using one
  desktop PC hardware configuration, the query on the FTS3 table returns in
  approximately 0.03 seconds, versus 22.5 for querying the ordinary table.

SELECT count(*) FROM enrondata1 WHERE content MATCH 'linux';  /* 0.03 seconds */
SELECT count(*) FROM enrondata2 WHERE content LIKE '%linux%'; /* 22.5 seconds */



  Of course, the two queries above are not entirely equivalent. For example
  the LIKE query matches rows that contain terms such as "linuxophobe"
  or "EnterpriseLinux" (as it happens, the Enron E-Mail Dataset does not
  actually contain any such terms), whereas the MATCH query on the FTS3 table
  selects only those rows that contain "linux" as a discrete token. Both
  searches are case-insensitive. The FTS3 table consumes around 2006 MB on
  disk compared to just 1453 MB for the ordinary table. Using the same
  hardware configuration used to perform the SELECT queries above, the FTS3
  table took just under 31 minutes to populate, versus 25 for the ordinary
  table.


c2: SQLite FTS3 and FTS4 Extensions
c3: 1.1. Differences between FTS3 and FTS4
c5: fts3.html#differences_between_fts3_and_fts4
c4:
FTS3 and FTS4 are nearly identical. They share most of their code in common,
  and their interfaces are the same. The differences are:


   FTS4 contains query performance optimizations that may significantly
       improve the performance of full-text queries that contain terms that are
       very common (present in a large percentage of table rows).

   FTS4 supports some additional options that may used with the matchinfo()
       function.

   Because it stores extra information on disk in two new
       shadow tables in order to support the performance
       optimizations and extra matchinfo() options, FTS4 tables may consume more
       disk space than the equivalent table created using FTS3. Usually the overhead
       is 1-2% or less, but may be as high as 10% if the documents stored in the
       FTS table are very small. The overhead may be reduced by specifying the
       directive "matchinfo=fts3" as part of the FTS4 table
       declaration, but this comes at the expense of sacrificing some of the
       extra supported matchinfo() options.

   FTS4 provides hooks (the compress and uncompress
       options) allowing data to be stored in a compressed
       form, reducing disk usage and IO.



  FTS4 is an enhancement to FTS3.
  FTS3 has been available since SQLite version 3.5.0 (2007-09-04)
  The enhancements for FTS4 were added with SQLite version 3.7.4
  (2010-12-07).


  Which module, FTS3 or FTS4, should you use in your application?  FTS4 is
  sometimes significantly faster than FTS3, even orders of magnitude faster
  depending on the query, though in the common case the performance of the two
  modules is similar. FTS4 also offers the enhanced matchinfo() outputs which
  can be useful in ranking the results of a MATCH operation.  On the
  other hand, in the absence of a matchinfo=fts3 directive FTS4 requires a little
  more disk space than FTS3, though only a percent of two in most cases.


  For newer applications, FTS4 is recommended; though if compatibility with older
  versions of SQLite is important, then FTS3 will usually serve just as well.


c2: SQLite FTS3 and FTS4 Extensions
c3: 1.2. Creating and Destroying FTS Tables
c5: fts3.html#creating_and_destroying_fts_tables
c4:
Like other virtual table types, new FTS tables are created using a
  CREATE VIRTUAL TABLE statement. The module name, which follows
  the USING keyword, is either "fts3" or "fts4". The virtual table module arguments may
  be left empty, in which case an FTS table with a single user-defined
  column named "content" is created. Alternatively, the module arguments
  may be passed a list of comma separated column names.


  If column names are explicitly provided for the FTS table as part of
  the CREATE VIRTUAL TABLE statement, then a datatype name may be optionally
  specified for each column. This is pure syntactic sugar, the
  supplied typenames are not used by FTS or the SQLite core for any
  purpose. The same applies to any constraints specified along with an
  FTS column name - they are parsed but not used or recorded by the system
  in any way.

-- Create an FTS table named "data" with one column - "content":
CREATE VIRTUAL TABLE data USING fts3();

-- Create an FTS table named "pages" with three columns:
CREATE VIRTUAL TABLE pages USING fts4(title, keywords, body);

-- Create an FTS table named "mail" with two columns. Datatypes
-- and column constraints are specified along with each column. These
-- are completely ignored by FTS and SQLite. 
CREATE VIRTUAL TABLE mail USING fts3(
  subject VARCHAR(256) NOT NULL,
  body TEXT CHECK(length(body)&lt;10240)
);



  As well as a list of columns, the module arguments passed to a CREATE
  VIRTUAL TABLE statement used to create an FTS table may be used to specify
  a tokenizer. This is done by specifying a string of the form
  "tokenize=&lt;tokenizer name&gt; &lt;tokenizer args&gt;" in place of a column
  name, where &lt;tokenizer name&gt; is the name of the tokenizer to use and
  &lt;tokenizer args&gt; is an optional list of whitespace separated qualifiers
  to pass to the tokenizer implementation. A tokenizer specification may be
  placed anywhere in the column list, but at most one tokenizer declaration is
  allowed for each CREATE VIRTUAL TABLE statement. See below for a
  detailed description of using (and, if necessary, implementing) a tokenizer.

-- Create an FTS table named "papers" with two columns that uses
-- the tokenizer "porter".
CREATE VIRTUAL TABLE papers USING fts3(author, document, tokenize=porter);

-- Create an FTS table with a single column - "content" - that uses
-- the "simple" tokenizer.
CREATE VIRTUAL TABLE data USING fts4(tokenize=simple);

-- Create an FTS table with two columns that uses the "icu" tokenizer.
-- The qualifier "en_AU" is passed to the tokenizer implementation
CREATE VIRTUAL TABLE names USING fts3(a, b, tokenize=icu en_AU);



  FTS tables may be dropped from the database using an ordinary DROP TABLE
  statement. For example:

-- Create, then immediately drop, an FTS4 table.
CREATE VIRTUAL TABLE data USING fts4();
DROP TABLE data;


c2: SQLite FTS3 and FTS4 Extensions
c3: 1.3. Populating FTS Tables
c5: fts3.html#populating_fts_tables
c4:
FTS tables are populated using INSERT, UPDATE and DELETE
    statements in the same way as ordinary SQLite tables are.

  
    As well as the columns named by the user (or the "content" column if no
    module arguments were specified as part of the CREATE VIRTUAL TABLE
    statement), each FTS table has a "rowid" column. The rowid of an FTS
    table behaves in the same way as the rowid column of an ordinary SQLite
    table, except that the values stored in the rowid column of an FTS table
    remain unchanged if the database is rebuilt using the VACUUM command.
    For FTS tables, "docid" is allowed as an alias along with the usual "rowid",
    "oid" and "_oid_" identifiers. Attempting to insert or update a row with a
    docid value that already exists in the table is an error, just as it would
    be with an ordinary SQLite table.

  
    There is one other subtle difference between "docid" and the normal SQLite
    aliases for the rowid column. Normally, if an INSERT or UPDATE statement
    assigns discrete values to two or more aliases of the rowid column, SQLite
    writes the rightmost of such values specified in the INSERT or UPDATE
    statement to the database. However, assigning a non-NULL value to both
    the "docid" and one or more of the SQLite rowid aliases when inserting or
    updating an FTS table is considered an error. See below for an example.

-- Create an FTS table
CREATE VIRTUAL TABLE pages USING fts4(title, body);

-- Insert a row with a specific docid value.
INSERT INTO pages(docid, title, body) VALUES(53, 'Home Page', 'SQLite is a software...');

-- Insert a row and allow FTS to assign a docid value using the same algorithm as
-- SQLite uses for ordinary tables. In this case the new docid will be 54,
-- one greater than the largest docid currently present in the table.
INSERT INTO pages(title, body) VALUES('Download', 'All SQLite source code...');

-- Change the title of the row just inserted.
UPDATE pages SET title = 'Download SQLite' WHERE rowid = 54;

-- Delete the entire table contents.
DELETE FROM pages;

-- The following is an error. It is not possible to assign non-NULL values to both
-- the rowid and docid columns of an FTS table.
INSERT INTO pages(rowid, docid, title, body) VALUES(1, 2, 'A title', 'A document body');


  
    To support full-text queries, FTS maintains an inverted index that maps
    from each unique term or word that appears in the dataset to the locations
    in which it appears within the table contents. For the curious, a
    complete description of the data structure used to store
    this index within the database file appears below. A feature of
    this data structure is that at any time the database may contain not
    one index b-tree, but several different b-trees that are incrementally
    merged as rows are inserted, updated and deleted. This technique improves
    performance when writing to an FTS table, but causes some overhead for
    full-text queries that use the index. Evaluating the special "optimize" command,
    an SQL statement of the
    form "INSERT INTO &lt;fts-table&gt;(&lt;fts-table&gt;) VALUES('optimize')",
    causes FTS to merge all existing index b-trees into a single large
    b-tree containing the entire index. This can be an expensive operation,
    but may speed up future queries.

  
    For example, to optimize the full-text index for an FTS table named
    "docs":

-- Optimize the internal structure of FTS table "docs".
INSERT INTO docs(docs) VALUES('optimize');


  
    The statement above may appear syntactically incorrect to some. Refer to
    the section describing the simple fts queries for an explanation.

  
    There is another, deprecated, method for invoking the optimize
    operation using a SELECT statement. New code should use statements
    similar to the INSERT above to optimize FTS structures.


c2: SQLite FTS3 and FTS4 Extensions
c3: 1.4. Simple FTS Queries
c5: fts3.html#simple_fts_queries
c4:
As for all other SQLite tables, virtual or otherwise, data is retrieved
  from FTS tables using a SELECT statement.


  FTS tables can be queried efficiently using SELECT statements of two
  different forms:


  
    Query by rowid. If the WHERE clause of the SELECT statement
    contains a sub-clause of the form "rowid = ?", where ? is an SQL expression,
    FTS is able to retrieve the requested row directly using the equivalent
    of an SQLite INTEGER PRIMARY KEY index.

  
    Full-text query. If the WHERE clause of the SELECT statement contains
    a sub-clause of the form "&lt;column&gt; MATCH ?", FTS is able to use
    the built-in full-text index to restrict the search to those documents
    that match the full-text query string specified as the right-hand operand
    of the MATCH clause.



  If neither of these two query strategies can be used, all
  queries on FTS tables are implemented using a linear scan of the entire
  table. If the table contains large amounts of data, this may be an
  impractical approach (the first example on this page shows that a linear
  scan of 1.5 GB of data takes around 30 seconds using a modern PC).

-- The examples in this block assume the following FTS table:
CREATE VIRTUAL TABLE mail USING fts3(subject, body);

SELECT * FROM mail WHERE rowid = 15;                -- Fast. Rowid lookup.
SELECT * FROM mail WHERE body MATCH 'sqlite';       -- Fast. Full-text query.
SELECT * FROM mail WHERE mail MATCH 'search';       -- Fast. Full-text query.
SELECT * FROM mail WHERE rowid BETWEEN 15 AND 20;   -- Fast. Rowid lookup.
SELECT * FROM mail WHERE subject = 'database';      -- Slow. Linear scan.
SELECT * FROM mail WHERE subject MATCH 'database';  -- Fast. Full-text query.



  In all of the full-text queries above, the right-hand operand of the MATCH
  operator is a string consisting of a single term. In this case, the MATCH
  expression evaluates to true for all documents that contain one or more
  instances of the specified word ("sqlite", "search" or "database", depending
  on which example you look at). Specifying a single term as the right-hand
  operand of the MATCH operator results in the simplest and most common type
  of full-text query possible. However more complicated queries are possible,
  including phrase searches, term-prefix searches and searches for documents
  containing combinations of terms occurring within a defined proximity of each
  other. The various ways in which the full-text index may be queried are
  described below.


  Normally, full-text queries are case-insensitive. However, this
  is dependent on the specific tokenizer used by the FTS table
  being queried. Refer to the section on tokenizers for details.


  The paragraph above notes that a MATCH operator with a simple term as the
  right-hand operand evaluates to true for all documents that contain the
  specified term. In this context, the "document" may refer to either the
  data stored in a single column of a row of an FTS table, or to the contents
  of all columns in a single row, depending on the identifier used as the
  left-hand operand to the MATCH operator. If the identifier specified as
  the left-hand operand of the MATCH operator is an FTS table column name,
  then the document that the search term must be contained in is the value
  stored in the specified column. However, if the identifier is the name
  of the FTS table itself, then the MATCH operator evaluates to true
  for each row of the FTS table for which any column contains the search
  term. The following example demonstrates this:

-- Example schema
CREATE VIRTUAL TABLE mail USING fts3(subject, body);

-- Example table population
INSERT INTO mail(docid, subject, body) VALUES(1, 'software feedback', 'found it too slow');
INSERT INTO mail(docid, subject, body) VALUES(2, 'software feedback', 'no feedback');
INSERT INTO mail(docid, subject, body) VALUES(3, 'slow lunch order',  'was a software problem');

-- Example queries
SELECT * FROM mail WHERE subject MATCH 'software';    -- Selects rows 1 and 2
SELECT * FROM mail WHERE body    MATCH 'feedback';    -- Selects row 2
SELECT * FROM mail WHERE mail    MATCH 'software';    -- Selects rows 1, 2 and 3
SELECT * FROM mail WHERE mail    MATCH 'slow';        -- Selects rows 1 and 3



  At first glance, the final two full-text queries in the example above seem
  to be syntactically incorrect, as there is a table name ("mail") used as
  an SQL expression. The reason this is acceptable is that each FTS table
  actually has a HIDDEN column with the same name
  as the table itself (in this case, "mail"). The value stored in this
  column is not meaningful to the application, but can be used as the
  left-hand operand to a MATCH operator. This special column may also be
  passed as an argument to the FTS auxiliary functions.


  The following example illustrates the above. The expressions "docs",
  "docs.docs" and "main.docs.docs" all refer to column "docs". However, the
  expression "main.docs" does not refer to any column. It could be used to
  refer to a table, but a table name is not allowed in the context in which
  it is used below.

-- Example schema
CREATE VIRTUAL TABLE docs USING fts4(content);

-- Example queries
SELECT * FROM docs WHERE docs MATCH 'sqlite';              -- OK.
SELECT * FROM docs WHERE docs.docs MATCH 'sqlite';         -- OK.
SELECT * FROM docs WHERE main.docs.docs MATCH 'sqlite';    -- OK.
SELECT * FROM docs WHERE main.docs MATCH 'sqlite';         -- Error.


c2: SQLite FTS3 and FTS4 Extensions
c3: 1.5. Summary
c5: fts3.html#summary
c4:
From the users point of view, FTS tables are similar to ordinary SQLite
  tables in many ways. Data may be added to, modified within and removed
  from FTS tables using the INSERT, UPDATE and DELETE commands just as
  it may be with ordinary tables. Similarly, the SELECT command may be used
  to query data. The following list summarizes the differences between FTS
  and ordinary tables:


  
    As with all virtual table types, it is not possible to create indices or
    triggers attached to FTS tables. Nor is it possible to use the ALTER TABLE
    command to add extra columns to FTS tables (although it is possible to use
    ALTER TABLE to rename an FTS table).

  
    Data-types specified as part of the "CREATE VIRTUAL TABLE" statement
    used to create an FTS table are ignored completely. Instead of the
    normal rules for applying type affinity to inserted values, all
    values inserted into FTS table columns (except the special rowid
    column) are converted to type TEXT before being stored.

  
    FTS tables permit the special alias "docid" to be used to refer to the
    rowid column supported by all virtual tables.

  
    The FTS MATCH operator is supported for queries based on the built-in
    full-text index.

  
    The FTS auxiliary functions, snippet(), offsets(), and matchinfo() are
    available to support full-text queries.

  
    

    Every FTS table has a hidden column with the
    same name as the table itself. The value contained in each row for the
    hidden column is a blob that is only useful as the left operand of a
    MATCH operator, or as the left-most argument to one
    of the FTS auxiliary functions.


c2: SQLite FTS3 and FTS4 Extensions
c3: 2. Compiling and Enabling FTS3 and FTS4
c5: fts3.html#compiling_and_enabling_fts3_and_fts4
c4:
Although FTS3 and FTS4 are included with the SQLite core source code, they are not
  enabled by default. To build SQLite with FTS functionality enabled, define
  the preprocessor macro SQLITE_ENABLE_FTS3 when compiling. New applications
  should also define the SQLITE_ENABLE_FTS3_PARENTHESIS macro to enable the
  enhanced query syntax (see below). Usually, this is done by adding the
  following two switches to the compiler command line:

-DSQLITE_ENABLE_FTS3
-DSQLITE_ENABLE_FTS3_PARENTHESIS



  Note that enabling FTS3 also makes FTS4 available.  There is not a separate
  SQLITE_ENABLE_FTS4 compile-time option.  A build of SQLite either supports
  both FTS3 and FTS4 or it supports neither.


  If using the amalgamation autoconf based build system, setting the CPPFLAGS
  environment variable while running the 'configure' script is an easy
  way to set these macros. For example, the following command:

CPPFLAGS="-DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS3_PARENTHESIS" ./configure &lt;configure options&gt;



  where &lt;configure options&gt; are those options normally passed to
  the configure script, if any.


  Because FTS3 and FTS4 are virtual tables, The SQLITE_ENABLE_FTS3 compile-time option
  is incompatible with the SQLITE_OMIT_VIRTUALTABLE option.


  If a build of SQLite does not include the FTS modules, then any attempt to prepare an
  SQL statement to create an FTS3 or FTS4 table or to drop or access an existing
  FTS table in any way will fail. The error message returned will be similar
  to "no such module: ftsN" (where N is either 3 or 4).


  If the C version of the ICU library
  is available, then FTS may also be compiled with the SQLITE_ENABLE_ICU
  pre-processor macro defined. Compiling with this macro enables an FTS
  tokenizer that uses the ICU library to split a document into terms
  (words) using the conventions for a specified language and locale.

-DSQLITE_ENABLE_ICU


c2: SQLite FTS3 and FTS4 Extensions
c3: 3. Full-text Index Queries
c5: fts3.html#full_text_index_queries
c4:
The most useful thing about FTS tables is the queries that may be
  performed using the built-in full-text index. Full-text queries are
  performed by specifying a clause of the form
  "&lt;column&gt; MATCH &lt;full-text query expression&gt;" as part of the WHERE
  clause of a SELECT statement that reads data from an FTS table.
  Simple FTS queries that return all documents that
  contain a given term are described above. In that discussion the right-hand
  operand of the MATCH operator was assumed to be a string consisting of a
  single term. This section describes the more complex query types supported
  by FTS tables, and how they may be utilized by specifying a more
  complex query expression as the right-hand operand of a MATCH operator.


  FTS tables support three basic query types:


  

  Token or token prefix queries.
    An FTS table may be queried for all documents that contain a specified
    term (the simple case described above), or for
    all documents that contain a term with a specified prefix. As we have
    seen, the query expression for a specific term is simply the term itself.
    The query expression used to search for a term prefix is the prefix
    itself with a '*' character appended to it. For example:


-- Virtual table declaration
CREATE VIRTUAL TABLE docs USING fts3(title, body);

-- Query for all documents containing the term "linux":
SELECT * FROM docs WHERE docs MATCH 'linux';

-- Query for all documents containing a term with the prefix "lin". This will match
-- all documents that contain "linux", but also those that contain terms "linear",
--"linker", "linguistic" and so on.
SELECT * FROM docs WHERE docs MATCH 'lin*';



  
    Normally, a token or token prefix query is matched against the FTS table
    column specified as the left-hand side of the MATCH operator. Or, if the
    special column with the same name as the FTS table itself is specified,
    against all columns. This may be overridden by specifying a column-name
    followed by a ":" character before a basic term query. There may be space
    between the ":" and the term to query for, but not between the column-name
    and the ":" character. For example:


-- Query the database for documents for which the term "linux" appears in
-- the document title, and the term "problems" appears in either the title
-- or body of the document.
SELECT * FROM docs WHERE docs MATCH 'title:linux problems';

-- Query the database for documents for which the term "linux" appears in
-- the document title, and the term "driver" appears in the body of the document
-- ("driver" may also appear in the title, but this alone will not satisfy the
-- query criteria).
SELECT * FROM docs WHERE body MATCH 'title:linux driver';



  
    If the FTS table is an FTS4 table (not FTS3), a token may also be prefixed
    with a "&#94;" character. In this case, in order to match the token must
    appear as the very first token in any column of the matching row. Examples:


-- All documents for which "linux" is the first token of at least one
-- column.
SELECT * FROM docs WHERE docs MATCH '&#94;linux';

-- All documents for which the first token in column "title" begins with "lin".
SELECT * FROM docs WHERE body MATCH 'title: &#94;lin*';





  Phrase queries.
    A phrase query is a query that retrieves all documents that contain a
    nominated set of terms or term prefixes in a specified order with no
    intervening tokens. Phrase queries are specified by enclosing a space
    separated sequence of terms or term prefixes in double quotes (").
    For example:


-- Query for all documents that contain the phrase "linux applications".
SELECT * FROM docs WHERE docs MATCH '"linux applications"';

-- Query for all documents that contain a phrase that matches "lin* app*". As well as
-- "linux applications", this will match common phrases such as "linoleum appliances"
-- or "link apprentice".
SELECT * FROM docs WHERE docs MATCH '"lin* app*"';





  NEAR queries.
    A NEAR query is a query that returns documents that contain a two or
    more nominated terms or phrases within a specified proximity of each
    other (by default with 10 or less intervening terms). A NEAR query is
    specified by putting the keyword "NEAR" between two phrase, token or
    token prefix queries. To specify a proximity other than the default,
    an operator of the form "NEAR/&lt;N&gt;" may be used, where
    &lt;N&gt; is the maximum number of intervening terms allowed.
    For example:


-- Virtual table declaration.
CREATE VIRTUAL TABLE docs USING fts4();

-- Virtual table data.
INSERT INTO docs VALUES('SQLite is an ACID compliant embedded relational database management system');

-- Search for a document that contains the terms "sqlite" and "database" with
-- not more than 10 intervening terms. This matches the only document in
-- table docs (since there are only six terms between "SQLite" and "database"
-- in the document).
SELECT * FROM docs WHERE docs MATCH 'sqlite NEAR database';

-- Search for a document that contains the terms "sqlite" and "database" with
-- not more than 6 intervening terms. This also matches the only document in
-- table docs. Note that the order in which the terms appear in the document
-- does not have to be the same as the order in which they appear in the query.
SELECT * FROM docs WHERE docs MATCH 'database NEAR/6 sqlite';

-- Search for a document that contains the terms "sqlite" and "database" with
-- not more than 5 intervening terms. This query matches no documents.
SELECT * FROM docs WHERE docs MATCH 'database NEAR/5 sqlite';

-- Search for a document that contains the phrase "ACID compliant" and the term
-- "database" with not more than 2 terms separating the two. This matches the
-- document stored in table docs.
SELECT * FROM docs WHERE docs MATCH 'database NEAR/2 "ACID compliant"';

-- Search for a document that contains the phrase "ACID compliant" and the term
-- "sqlite" with not more than 2 terms separating the two. This also matches
-- the only document stored in table docs.
SELECT * FROM docs WHERE docs MATCH '"ACID compliant" NEAR/2 sqlite';



  
    More than one NEAR operator may appear in a single query. In this case each
    pair of terms or phrases separated by a NEAR operator must appear within the
    specified proximity of each other in the document. Using the same table and
    data as in the block of examples above:


-- The following query selects documents that contains an instance of the term 
-- "sqlite" separated by two or fewer terms from an instance of the term "acid",
-- which is in turn separated by two or fewer terms from an instance of the term
-- "relational".
SELECT * FROM docs WHERE docs MATCH 'sqlite NEAR/2 acid NEAR/2 relational';

-- This query matches no documents. There is an instance of the term "sqlite" with
-- sufficient proximity to an instance of "acid" but it is not sufficiently close
-- to an instance of the term "relational".
SELECT * FROM docs WHERE docs MATCH 'acid NEAR/2 sqlite NEAR/2 relational';



  Phrase and NEAR queries may not span multiple columns within a row.


  The three basic query types described above may be used to query the full-text
  index for the set of documents that match the specified criteria. Using the
  FTS query expression language it is possible to perform various set
  operations on the results of basic queries. There are currently three
  supported operations:


   The AND operator determines the intersection of two sets of documents.

   The OR operator calculates the union of two sets of documents.

   The NOT operator (or, if using the standard syntax, a unary "-" operator)
       may be used to compute the relative complement of one set of
       documents with respect to another.



  The FTS modules may be compiled to use one of two slightly different versions
  of the full-text query syntax, the "standard" query syntax and the "enhanced"
  query syntax. The basic term, term-prefix, phrase and NEAR queries described
  above are the same in both versions of the syntax. The way in which set
  operations are specified is slightly different. The following two sub-sections
  describe the part of the two query syntaxes that pertains to set operations.
  Refer to the description of how to compile fts for compilation notes.


c2: SQLite FTS3 and FTS4 Extensions
c3: 3.1. Set Operations Using The Enhanced Query Syntax
c5: fts3.html#_set_operations_using_the_enhanced_query_syntax
c4:
The enhanced query syntax supports the AND, OR and NOT binary set operators.
  Each of the two operands to an operator may be a basic FTS query, or the
  result of another AND, OR&nbsp;or NOT set operation. Operators must be entered
  using capital letters. Otherwise, they are interpreted as basic term queries
  instead of set operators.


  The AND operator may be implicitly specified. If two basic queries appear
  with no operator separating them in an FTS query string, the results are
  the same as if the two basic queries were separated by an AND operator.
  For example, the query expression "implicit operator" is a more succinct
  version of "implicit AND operator".

-- Virtual table declaration
CREATE VIRTUAL TABLE docs USING fts3();

-- Virtual table data
INSERT INTO docs(docid, content) VALUES(1, 'a database is a software system');
INSERT INTO docs(docid, content) VALUES(2, 'sqlite is a software system');
INSERT INTO docs(docid, content) VALUES(3, 'sqlite is a database');

-- Return the set of documents that contain the term "sqlite", and the
-- term "database". This query will return the document with docid 3 only.
SELECT * FROM docs WHERE docs MATCH 'sqlite AND database';

-- Again, return the set of documents that contain both "sqlite" and
-- "database". This time, use an implicit AND operator. Again, document
-- 3 is the only document matched by this query. 
SELECT * FROM docs WHERE docs MATCH 'database sqlite';

-- Query for the set of documents that contains either "sqlite" or "database".
-- All three documents in the database are matched by this query.
SELECT * FROM docs WHERE docs MATCH 'sqlite OR database';

-- Query for all documents that contain the term "database", but do not contain
-- the term "sqlite". Document 1 is the only document that matches this criteria.
SELECT * FROM docs WHERE docs MATCH 'database NOT sqlite';

-- The following query matches no documents. Because "and" is in lowercase letters,
-- it is interpreted as a basic term query instead of an operator. Operators must
-- be specified using capital letters. In practice, this query will match any documents
-- that contain each of the three terms "database", "and" and "sqlite" at least once.
-- No documents in the example data above match this criteria.
SELECT * FROM docs WHERE docs MATCH 'database and sqlite';



  The examples above all use basic full-text term queries as both operands of
  the set operations demonstrated. Phrase and NEAR queries may also be used,
  as may the results of other set operations. When more than one set operation
  is present in an FTS query, the precedence of operators is as follows:


  OperatorEnhanced Query Syntax Precedence
  NOT  Highest precedence (tightest grouping).
  AND 
  OR   Lowest precedence (loosest grouping).



  When using the enhanced query syntax, parenthesis may be used to override
  the default precedence of the various operators. For example:

-- Return the docid values associated with all documents that contain the
-- two terms "sqlite" and "database", and/or contain the term "library".
SELECT docid FROM docs WHERE docs MATCH 'sqlite AND database OR library';

-- This query is equivalent to the above.
SELECT docid FROM docs WHERE docs MATCH 'sqlite AND database'
  UNION
SELECT docid FROM docs WHERE docs MATCH 'library';

-- Query for the set of documents that contains the term "linux", and at least
-- one of the phrases "sqlite database" and "sqlite library".
SELECT docid FROM docs WHERE docs MATCH '("sqlite database" OR "sqlite library") AND linux';

-- This query is equivalent to the above.
SELECT docid FROM docs WHERE docs MATCH 'linux'
  INTERSECT
SELECT docid FROM (
  SELECT docid FROM docs WHERE docs MATCH '"sqlite library"'
    UNION
  SELECT docid FROM docs WHERE docs MATCH '"sqlite database"'
);


c2: SQLite FTS3 and FTS4 Extensions
c3: 3.2. Set Operations Using The Standard Query Syntax
c5: fts3.html#set_operations_using_the_standard_query_syntax
c4:
FTS query set operations using the standard query syntax are similar, but
  not identical, to set operations with the enhanced query syntax. There
  are four differences, as follows:


   Only the implicit version of the AND operator is supported.
    Specifying the string "AND" as part of a standard query syntax query is
    interpreted as a term query for the set of documents containing the term
    "and".



   Parenthesis are not supported.



   The NOT operator is not supported. Instead of the NOT
    operator, the standard query syntax supports a unary "-" operator that
    may be applied to basic term and term-prefix queries (but not to phrase
    or NEAR queries). A term or term-prefix that has a unary "-" operator
    attached to it may not appear as an operand to an OR operator. An FTS
    query may not consist entirely of terms or term-prefix queries with unary
    "-" operators attached to them.


-- Search for the set of documents that contain the term "sqlite" but do
-- not contain the term "database".
SELECT * FROM docs WHERE docs MATCH 'sqlite -database';



   The relative precedence of the set operations is different.
   In particular, using the standard query syntax the "OR" operator has a
   higher precedence than "AND". The precedence of operators when using the
   standard query syntax is:



  OperatorStandard Query Syntax Precedence
  Unary "-"  Highest precedence (tightest grouping).
  OR  
  AND  Lowest precedence (loosest grouping).



  The following example illustrates precedence of operators using the standard
  query syntax:


-- Search for documents that contain at least one of the terms "database"
-- and "sqlite", and also contain the term "library". Because of the differences
-- in operator precedences, this query would have a different interpretation using
-- the enhanced query syntax.
SELECT * FROM docs WHERE docs MATCH 'sqlite OR database library';


c2: SQLite FTS3 and FTS4 Extensions
c3: 4. Auxiliary Functions - Snippet, Offsets and Matchinfo
c5: fts3.html#auxiliary_functions_snippet_offsets_and_matchinfo
c4:
The FTS3 and FTS4 modules provide three special SQL scalar functions that may be useful
  to the developers of full-text query systems: "snippet", "offsets" and
  "matchinfo". The purpose of the "snippet" and "offsets" functions is to allow
  the user to identify the location of queried terms in the returned documents.
  The "matchinfo" function provides the user with metrics that may be useful
  for filtering or sorting query results according to relevance.


  The first argument to all three special SQL scalar functions
  must be the FTS hidden column of the FTS table that the function is
  applied to.  The FTS hidden column is an automatically-generated column found on
  all FTS tables that has the same name as the FTS table itself.
  For example, given an FTS table named "mail":

SELECT offsets(mail) FROM mail WHERE mail MATCH &lt;full-text query expression&gt;;
SELECT snippet(mail) FROM mail WHERE mail MATCH &lt;full-text query expression&gt;;
SELECT matchinfo(mail) FROM mail WHERE mail MATCH &lt;full-text query expression&gt;;



  The three auxiliary functions are only useful within a SELECT statement that
  uses the FTS table's full-text index. If used within a SELECT that uses
  the "query by rowid" or "linear scan" strategies, then the snippet and
  offsets both return an empty string, and the matchinfo function returns
  a blob value zero bytes in size.




  All three auxiliary functions extract a set of "matchable phrases" from
  the FTS query expression to work with. The set of matchable phrases for
  a given query consists of all phrases (including unquoted tokens and
  token prefixes) in the expression except those that are prefixed with
  a unary "-" operator (standard syntax) or are part of a sub-expression
  that is used as the right-hand operand of a NOT operator.


  With the following provisos, each series of tokens in the FTS table that
  matches one of the matchable phrases in the query expression is known as a
  "phrase match":


   If a matchable phrase is part of a series of phrases connected by
       NEAR operators in the FTS query expression, then each phrase match
       must be sufficiently close to other phrase matches of the relevant
       types to satisfy the NEAR condition.

   If the matchable phrase in the FTS query is restricted to matching
       data in a specified FTS table column, then only phrase matches that
       occur within that column are considered.


c2: SQLite FTS3 and FTS4 Extensions
c3: 4.1. The Offsets Function
c5: fts3.html#the_offsets_function
c4:
For a SELECT query that uses the full-text index, the offsets() function
  returns a text value containing a series of space-separated integers. For
  each term in each phrase match of the current row,
  there are four integers in the returned list. Each set of four integers is
  interpreted as follows:


  Integer Interpretation
  0
      The column number that the term instance occurs in (0 for the
          leftmost column of the FTS table, 1 for the next leftmost, etc.).
  1
      The term number of the matching term within the full-text query
          expression. Terms within a query expression are numbered starting
          from 0 in the order that they occur.
  2
      The byte offset of the matching term within the column.
  3
      The size of the matching term in bytes.



  The following block contains examples that use the offsets function.

CREATE VIRTUAL TABLE mail USING fts3(subject, body);
INSERT INTO mail VALUES('hello world', 'This message is a hello world message.');
INSERT INTO mail VALUES('urgent: serious', 'This mail is seen as a more serious mail');

-- The following query returns a single row (as it matches only the first
-- entry in table "mail". The text returned by the offsets function is
-- "0 0 6 5 1 0 24 5".
--
-- The first set of four integers in the result indicate that column 0
-- contains an instance of term 0 ("world") at byte offset 6. The term instance
-- is 5 bytes in size. The second set of four integers shows that column 1
-- of the matched row contains an instance of term 0 ("world") at byte offset
-- 24. Again, the term instance is 5 bytes in size.
SELECT offsets(mail) FROM mail WHERE mail MATCH 'world';

-- The following query returns also matches only the first row in table "mail".
-- In this case the returned text is "1 0 5 7 1 0 30 7".
SELECT offsets(mail) FROM mail WHERE mail MATCH 'message';

-- The following query matches the second row in table "mail". It returns the
-- text "1 0 28 7 1 1 36 4". Only those occurrences of terms "serious" and "mail"
-- that are part of an instance of the phrase "serious mail" are identified; the
-- other occurrences of "serious" and "mail" are ignored.
SELECT offsets(mail) FROM mail WHERE mail MATCH '"serious mail"';


c2: SQLite FTS3 and FTS4 Extensions
c3: 4.2. The Snippet Function
c5: fts3.html#the_snippet_function
c4:
The snippet function is used to create formatted fragments of document text
  for display as part of a full-text query results report. The snippet function
  may be passed between one and six arguments, as follows:


  Argument Default Value Description
  0 N/A
       The first argument to the snippet function must always be the FTS hidden column
           of the FTS table being queried and from which the snippet is to be taken.  The
           FTS hidden column is an automatically generated column with the same name as the
           FTS table itself.
  1 "&lt;b&gt;"
       The "start match" text.
  2 "&lt;/b&gt;"
       The "end match" text.
  3 "&lt;b&gt;...&lt;/b&gt;"
       The "ellipses" text.
  4 -1
       The FTS table column number to extract the returned fragments of
           text from. Columns are numbered from left to right starting with
           zero. A negative value indicates that the text may be extracted
           from any column.
  5 -15
       The absolute value of this integer argument is used as the
           (approximate) number of tokens to include in the returned text
           value. The maximum allowable absolute value is 64. The value of
           this argument is referred to as N in the discussion below.



  The snippet function first attempts to find a fragment of text consisting
  of |N| tokens within the current row that contains at least one phrase
  match for each matchable phrase matched somewhere in the current row,
  where |N| is the absolute value of the sixth argument passed to the
  snippet function. If the text stored in a single column contains less than
  |N| tokens, then the entire column value is considered. Text fragments
  may not span multiple columns.


  If such a text fragment can be found, it is returned with the following
  modifications:


   If the text fragment does not begin at the start of a column value,
       the "ellipses" text is prepended to it.
   If the text fragment does not finish at the end of a column value,
       the "ellipses" text is appended to it.
   For each token in the text fragment that is part of a phrase match,
       the "start match" text is inserted into the fragment before the token,
       and the "end match" text is inserted immediately after it.



  If more than one such fragment can be found, then fragments that contain
  a larger number of "extra" phrase matches are favored. The start of
  the selected text fragment may be moved a few tokens forward or backward
  to attempt to concentrate the phrase matches toward the center of the
  fragment.


  Assuming N is a positive value, if no fragments can be found that
  contain a phrase match corresponding to each matchable phrase, the snippet
  function attempts to find two fragments of approximately N/2 tokens
  that between them contain at least one phrase match for each matchable phrase
  matched by the current row. If this fails, attempts are made to find three
  fragments of N/3 tokens each and finally four N/4 token
  fragments. If a set of four fragments cannot be found that encompasses the
  required phrase matches, the four fragments of N/4 tokens that provide
  the best coverage are selected.


  If N is a negative value, and no single fragment can be found
  containing the required phrase matches, the snippet function searches
  for two fragments of |N| tokens each, then three, then four. In
  other words, if the specified value of N is negative, the sizes
  of the fragments is not decreased if more than one fragment is required
  to provide the desired phrase match coverage.


  After the M fragments have been located, where M is between
  two and four as described in the paragraphs above, they are joined together
  in sorted order with the "ellipses" text separating them. The three
  modifications enumerated earlier are performed on the text before it is
  returned.

Note: In this block of examples, newlines and whitespace characters have
been inserted into the document inserted into the FTS table, and the expected
results described in SQL comments. This is done to enhance readability only,
they would not be present in actual SQLite commands or output.

-- Create and populate an FTS table.
CREATE VIRTUAL TABLE text USING fts4();
INSERT INTO text VALUES('
  During 30 Nov-1 Dec, 2-3oC drops. Cool in the upper portion, minimum temperature 14-16oC
  and cool elsewhere, minimum temperature 17-20oC. Cold to very cold on mountaintops,
  minimum temperature 6-12oC. Northeasterly winds 15-30 km/hr. After that, temperature
  increases. Northeasterly winds 15-30 km/hr.
');

-- The following query returns the text value:
--
--   "&lt;b&gt;...&lt;/b&gt;cool elsewhere, minimum temperature 17-20oC. &lt;b&gt;Cold&lt;/b&gt; to very 
--    &lt;b&gt;cold&lt;/b&gt; on mountaintops, minimum temperature 6&lt;b&gt;...&lt;/b&gt;".
--
SELECT snippet(text) FROM text WHERE text MATCH 'cold';

-- The following query returns the text value:
--
--   "...the upper portion, &#91;minimum&#93; &#91;temperature&#93; 14-16oC and cool elsewhere,
--    &#91;minimum&#93; &#91;temperature&#93; 17-20oC. Cold..."
--
SELECT snippet(text, '&#91;', '&#93;', '...') FROM text WHERE text MATCH '"min* tem*"'


c2: SQLite FTS3 and FTS4 Extensions
c3: 4.3. The Matchinfo Function
c5: fts3.html#matchinfo
c4:
The matchinfo function returns a blob value. If it is used within a query
  that does not use the full-text index (a "query by rowid" or "linear scan"),
  then the blob is zero bytes in size. Otherwise, the blob consists of zero
  or more 32-bit unsigned integers in machine byte-order. The exact number
  of integers in the returned array depends on both the query and the value
  of the second argument (if any) passed to the matchinfo function.


  The matchinfo function is called with either one or two arguments. As for
  all auxiliary functions, the first argument must be the special
  FTS hidden column. The second argument, if it is specified, must be a text value
  comprised only of the characters 'p', 'c', 'n', 'a', 'l', 's', 'x', 'y' and 'b'.
  If no second argument is explicitly supplied, it defaults to "pcx". The
  second argument is referred to as the "format string" below.


  Characters in the matchinfo format string are processed from left to right.
  Each character in the format string causes one or more 32-bit unsigned
  integer values to be added to the returned array. The "values" column in
  the following table contains the number of integer values appended to the
  output buffer for each supported format string character. In the formula
  given, cols is the number of columns in the FTS table, and
  phrases is the number of matchable phrases in
  the query.


  CharacterValuesDescription
  p 1 The number of matchable phrases in the query.
  c 1 The number of user defined columns in the FTS
    table (i.e. not including the docid or the FTS hidden column).
  x 3 * cols * phrases
    

      For each distinct combination of a phrase and table column, the
      following three values:
      
         In the current row, the number of times the phrase appears in
             the column.
         The total number of times the phrase appears in the column in
             all rows in the FTS table.
         The total number of rows in the FTS table for which the
             column contains at least one instance of the phrase.
      
      The first set of three values corresponds to the left-most column
      of the table (column 0) and the left-most matchable phrase in the
      query (phrase 0). If the table has more than one column, the second
      set of three values in the output array correspond to phrase 0 and
      column 1. Followed by phrase 0, column 2 and so on for all columns of
      the table. And so on for phrase 1, column 0, then phrase 1, column 1
      etc. In other words, the data for occurrences of phrase p in
      column c may be found using the following formula:

          hits_this_row  = array&#91;3 * (c + p*cols) + 0&#93;
          hits_all_rows  = array&#91;3 * (c + p*cols) + 1&#93;
          docs_with_hits = array&#91;3 * (c + p*cols) + 2&#93;

  ycols * phrases
    

      For each distinct combination of a phrase and table column, the
      number of usable phrase matches that appear in the column. This is
      usually identical to the first value in each set of three returned by the
      matchinfo 'x' flag. However, the number of hits reported by the
      'y' flag is zero for any phrase that is part of a sub-expression
      that does not match the current row. This makes a difference for
      expressions that contain AND operators that are descendants of OR
      operators. For example, consider the expression:

          a OR (b AND c)

      and the document:

          "a c d"

      The matchinfo 'x' flag would report a single hit for the phrases "a" and "c".
      However, the 'y' directive reports the number of hits for "c" as zero, as
      it is part of a sub-expression that does not match the document - (b AND c).
      For queries that do not contain AND operators descended from OR
      operators, the result values returned by 'y' are always the same as
      those returned by 'x'.


      The first value in the array of integer values corresponds to the
      leftmost column of the table (column 0) and the first phrase in the query
      (phrase 0). The values corresponding to other column/phrase combinations
      may be located using the following formula:


          hits_for_phrase_p_column_c  = array&#91;c + p*cols&#93;

      For queries that use OR expressions, or those that use LIMIT or return
      many rows, the 'y' matchinfo option may be faster than 'x'.

b((cols+31)/32) * phrases



  The matchinfo 'b' flag provides similar information to the
  matchinfo 'y' flag, but in a more
  compact form. Instead of the precise number of hits, 'b' provides a single
  boolean flag for each phrase/column combination. If the phrase is present in
  the column at least once (i.e. if the corresponding integer output of 'y' would
  be non-zero), the corresponding flag is set. Otherwise cleared.


  If the table has 32 or fewer columns, a single unsigned integer is output for
  each phrase in the query. The least significant bit of the integer is set if the
  phrase appears at least once in column 0. The second least significant bit is
  set if the phrase appears once or more in column 1. And so on.


  If the table has more than 32 columns, an extra integer is added to the output
  of each phrase for each extra 32 columns or part thereof. Integers
  corresponding to the same phrase are clumped together. For example, if a table
  with 45 columns is queried for two phrases, 4 integers are output. The first
  corresponds to phrase 0 and columns 0-31 of the table. The second integer
  contains data for phrase 0 and columns 32-44, and so on.


  For example, if nCol is the number of columns in the table, to determine if
  phrase p is present in column c:


    p_is_in_c = array&#91;p * ((nCol+31)/32)&#93; & (1 &lt;&lt; (c % 32))


  n 1 The number of rows in the FTS4 table. This value is
    only available when querying FTS4 tables, not FTS3.
  a cols For each column, the average number of
    tokens in the text values stored in the column (considering all rows in
    the FTS4 table). This value is only available when querying FTS4 tables,
    not FTS3.
  l cols 
    For each column, the length of the value stored in the current row of the
    FTS4 table, in tokens.  This value is only available when querying
    FTS4 tables, not FTS3. And only if the "matchinfo=fts3" directive was not
    specified as part of the "CREATE VIRTUAL TABLE" statement used to create
    the FTS4 table.
  s cols For each column, the length of the longest
    subsequence of phrase matches that the column value has in common
    with the query text. For example, if a table column contains the text
    'a b c d e' and the query is 'a c "d e"', then the length of the longest
    common subsequence is 2 (phrase "c" followed by phrase "d e").





  For example:

-- Create and populate an FTS4 table with two columns:
CREATE VIRTUAL TABLE t1 USING fts4(a, b);
INSERT INTO t1 VALUES('transaction default models default', 'Non transaction reads');
INSERT INTO t1 VALUES('the default transaction', 'these semantics present');
INSERT INTO t1 VALUES('single request', 'default data');

-- In the following query, no format string is specified and so it defaults
-- to "pcx". It therefore returns a single row consisting of a single blob
-- value 80 bytes in size (20 32-bit integers - 1 for "p", 1 for "c" and
-- 3*2*3 for "x"). If each block of 4 bytes in the blob is interpreted
-- as an unsigned integer in machine byte-order, the values will be:
--
--     3 2  1 3 2  0 1 1  1 2 2  0 1 1  0 0 0  1 1 1
--
-- The row returned corresponds to the second entry inserted into table t1.
-- The first two integers in the blob show that the query contained three
-- phrases and the table being queried has two columns. The next block of
-- three integers describes column 0 (in this case column "a") and phrase
-- 0 (in this case "default"). The current row contains 1 hit for "default"
-- in column 0, of a total of 3 hits for "default" that occur in column
-- 0 of any table row. The 3 hits are spread across 2 different rows.
--
-- The next set of three integers (0 1 1) pertain to the hits for "default"
-- in column 1 of the table (0 in this row, 1 in all rows, spread across 
-- 1 rows).
--
SELECT matchinfo(t1) FROM t1 WHERE t1 MATCH 'default transaction "these semantics"';

-- The format string for this query is "ns". The output array will therefore
-- contain 3 integer values - 1 for "n" and 2 for "s". The query returns
-- two rows (the first two rows in the table match). The values returned are:
--
--     3  1 1
--     3  2 0
--
-- The first value in the matchinfo array returned for both rows is 3 (the 
-- number of rows in the table). The following two values are the lengths 
-- of the longest common subsequence of phrase matches in each column.
SELECT matchinfo(t1, 'ns') FROM t1 WHERE t1 MATCH 'default transaction';



  The matchinfo function is much faster than either the snippet or offsets
  functions. This is because the implementation of both snippet and offsets
  is required to retrieve the documents being analyzed from disk, whereas
  all data required by matchinfo is available as part of the same portions
  of the full-text index that are required to implement the full-text query
  itself. This means that of the following two queries, the first may be
  an order of magnitude faster than the second:

SELECT docid, matchinfo(tbl) FROM tbl WHERE tbl MATCH &lt;query expression&gt;;
SELECT docid, offsets(tbl) FROM tbl WHERE tbl MATCH &lt;query expression&gt;;



  The matchinfo function provides all the information required to calculate
  probabilistic "bag-of-words" relevancy scores such as
  Okapi BM25/BM25F that may
  be used to order results in a full-text search application. Appendix A of this
  document, "search application tips", contains an example of using the
  matchinfo() function efficiently.


c2: SQLite FTS3 and FTS4 Extensions
c3: 5. Fts4aux - Direct Access to the Full-Text Index
c5: fts3.html#fts4aux
c4:
As of version 3.7.6 (2011-04-12),
  SQLite includes a new virtual table module called
  "fts4aux", which can be used to inspect the full-text index of an existing
  FTS table directly. Despite its name, fts4aux works just as well with FTS3
  tables as it does with FTS4 tables. Fts4aux tables are read-only. The only
  way to modify the contents of an fts4aux table is by modifying the
  contents of the associated FTS table. The fts4aux module is automatically
  included in all builds that include FTS.


  An fts4aux virtual table is constructed with one or two arguments.  When
  used with a single argument, that argument is the unqualified name of the
  FTS table that it will be used to access.  To access a table in a different
  database (for example, to create a TEMP fts4aux table that will access an
  FTS3 table in the MAIN database) use the two-argument form and give the
  name of the target database (ex: "main") in the first argument and the name
  of the FTS3/4 table as the second argument.  (The two-argument form of
  fts4aux was added for SQLite version 3.7.17 (2013-05-20)
  and will throw an error in prior releases.)
  For example:

-- Create an FTS4 table
CREATE VIRTUAL TABLE ft USING fts4(x, y);

-- Create an fts4aux table to access the full-text index for table "ft"
CREATE VIRTUAL TABLE ft_terms USING fts4aux(ft);

-- Create a TEMP fts4aux table accessing the "ft" table in "main"
CREATE VIRTUAL TABLE temp.ft_terms_2 USING fts4aux(main,ft);



  For each term present in the FTS table, there are between 2 and N+1 rows
  in the fts4aux table, where N is the number of user-defined columns in
  the associated FTS table. An fts4aux table always has the same four columns,
  as follows, from left to right:


  Column NameColumn Contents
  term
    Contains the text of the term for this row.
  col
    This column may contain either the text value '*' (i.e. a single
    character, U+002a) or an integer between 0 and N-1, where N is
    again the number of user-defined columns in the corresponding FTS table.

  documents
    This column always contains an integer value greater than zero.
    
    If the "col" column contains the value '*', then this column
    contains the number of rows of the FTS table that contain at least one
    instance of the term (in any column). If col contains an integer
    value, then this column contains the number of rows of the FTS table that
    contain at least one instance of the term in the column identified by
    the col value. As usual, the columns of the FTS table are numbered
    from left to right, starting with zero.

  occurrences
    This column also always contains an integer value greater than zero.
    
    If the "col" column contains the value '*', then this column
    contains the total number of instances of the term in all rows of the
    FTS table (in any column). Otherwise, if col contains an integer
    value, then this column contains the total number of instances of the
    term that appear in the FTS table column identified by the col
    value.

  languageid (hidden)
    

    This column determines which languageid is used to
    extract vocabulary from the FTS3/4 table.
    
    The default value for languageid is 0.  If an alternative language
    is specified in WHERE clause constraints, then that alternative is
    used instead of 0.  There can only be a single languageid per query.
    In other words, the WHERE clause cannot contain a range constraint
    or IN operator on the languageid.



  For example, using the tables created above:

INSERT INTO ft(x, y) VALUES('Apple banana', 'Cherry');
INSERT INTO ft(x, y) VALUES('Banana Date Date', 'cherry');
INSERT INTO ft(x, y) VALUES('Cherry Elderberry', 'Elderberry');

-- The following query returns this data:
--
--     apple       |  *  |  1  |  1
--     apple       |  0  |  1  |  1
--     banana      |  *  |  2  |  2
--     banana      |  0  |  2  |  2
--     cherry      |  *  |  3  |  3
--     cherry      |  0  |  1  |  1
--     cherry      |  1  |  2  |  2
--     date        |  *  |  1  |  2
--     date        |  0  |  1  |  2
--     elderberry  |  *  |  1  |  2
--     elderberry  |  0  |  1  |  1
--     elderberry  |  1  |  1  |  1
--
SELECT term, col, documents, occurrences FROM ft_terms;



  In the example, the values in the "term" column are all lower case,
  even though they were inserted into table "ft" in mixed case. This is because
  an fts4aux table contains the terms as extracted from the document text
  by the tokenizer. In this case, since table "ft" uses the
  simple tokenizer, this means all terms have been folded to
  lower case. Also, there is (for example) no row with column "term"
  set to "apple" and column "col" set to 1. Since there are no instances
  of the term "apple" in column 1, no row is present in the fts4aux table.


  During a transaction, some of the data written to an FTS table may be
  cached in memory and written to the database only when the transaction is
  committed. However the implementation of the fts4aux module is only able
  to read data from the database. In practice this means that if an fts4aux
  table is queried from within a transaction in which the associated
  FTS table has been modified, the results of the query are likely to reflect
  only a (possibly empty) subset of the changes made.


c2: SQLite FTS3 and FTS4 Extensions
c3: 6. FTS4 Options
c5: fts3.html#fts4_options
c4:
If the "CREATE VIRTUAL TABLE" statement specifies module FTS4 (not FTS3),
  then special directives - FTS4 options - similar to the "tokenize=*" option
  may also appear in place of column names. An FTS4 option consists of the
  option name, followed by an "=" character, followed by the option value.
  The option value may optionally be enclosed in single or double quotes, with
  embedded quote characters escaped in the same way as for SQL literals. There
  may not be whitespace on either side of the "=" character. For example,
  to create an FTS4 table with the value of option "matchinfo" set to "fts3":

-- Create a reduced-footprint FTS4 table.
CREATE VIRTUAL TABLE papers USING fts4(author, document, matchinfo=fts3);



  FTS4 currently supports the following options:


  OptionInterpretation
  compress
    The compress option is used to specify the compress function. It is an error to
    specify a compress function without also specifying an uncompress
    function. See below for details.

  content
    The content allows the text being indexed to be
    stored in a separate table distinct from the FTS4 table,
    or even outside of SQLite.

  languageid
    The languageid option causes the FTS4 table to have an additional hidden
    integer column that identifies the language of the text contained in
    each row.  The use of the languageid option allows the same FTS4 table
    to hold text in multiple languages or scripts, each with different tokenizer
    rules, and to query each language independently of the others.

  matchinfo
    When set to the value "fts3", the matchinfo option reduces the amount of
    information stored by FTS4 with the consequence that the "l" option of
    matchinfo() is no longer available.

  notindexed
    This option is used to specify the name of a column for which data is
    not indexed. Values stored in columns that are not indexed are not
    matched by MATCH queries. Nor are they recognized by auxiliary functions.
    A single CREATE VIRTUAL TABLE statement may have any number of notindexed
    options.

  order
    

    The "order" option may be set to either "DESC" or "ASC" (in upper or
    lower case). If it is set to "DESC", then FTS4 stores its data in such
    a way as to optimize returning results in descending order by docid.
    If it is set to "ASC" (the default), then the data structures are
    optimized for returning results in ascending order by docid.  In other
    words, if many of the queries run against the FTS4 table use "ORDER BY
    docid DESC", then it may improve performance to add the "order=desc"
    option to the CREATE VIRTUAL TABLE statement.

  prefix
    This option may be set to a comma-separated list of positive non-zero
    integers. For each integer N in the list, a separate index is created
    in the database file to optimize prefix queries where
    the query term is N bytes in length, not including the '*' character,
    when encoded using UTF-8. See below for details.

  uncompress
    This option is used to specify the uncompress function. It is an error to
    specify an uncompress function without also specifying a compress
    function. See below for details.



  When using FTS4, specifying a column name that contains an "=" character
  and is not either a "tokenize=*" specification or a recognized FTS4 option
  is an error. With FTS3, the first token in the unrecognized directive is
  interpreted as a column name. Similarly, specifying multiple "tokenize=*"
  directives in a single table declaration is an error when using FTS4, whereas
  the second and subsequent "tokenize=*" directives are interpreted as column
  names by FTS3. For example:

-- An error. FTS4 does not recognize the directive "xyz=abc".
CREATE VIRTUAL TABLE papers USING fts4(author, document, xyz=abc);

-- Create an FTS3 table with three columns - "author", "document"
-- and "xyz".
CREATE VIRTUAL TABLE papers USING fts3(author, document, xyz=abc);

-- An error. FTS4 does not allow multiple tokenize=* directives
CREATE VIRTUAL TABLE papers USING fts4(tokenize=porter, tokenize=simple);

-- Create an FTS3 table with a single column named "tokenize". The
-- table uses the "porter" tokenizer.
CREATE VIRTUAL TABLE papers USING fts3(tokenize=porter, tokenize=simple);

-- An error. Cannot create a table with two columns named "tokenize".
CREATE VIRTUAL TABLE papers USING fts3(tokenize=porter, tokenize=simple, tokenize=icu);


c2: SQLite FTS3 and FTS4 Extensions
c3: 6.1. The compress= and uncompress= options
c5: fts3.html#the_compress_and_uncompress_options
c4:
The compress and uncompress options allow FTS4 content to be stored in
  the database in a compressed form. Both options should be set to the name
  of an SQL scalar function registered using sqlite3_create_function()
  that accepts a single argument.


  The compress function should return a compressed version of the value
  passed to it as an argument. Each time data is written to the FTS4 table,
  each column value is passed to the compress function and the result value
  stored in the database. The compress function may return any type of SQLite
  value (blob, text, real, integer or null).


  The uncompress function should uncompress data previously compressed by
  the compress function. In other words, for all SQLite values X, it should
  be true that uncompress(compress(X)) equals X. When data that has been
  compressed by the compress function is read from the database by FTS4, it
  is passed to the uncompress function before it is used.


  If the specified compress or uncompress functions do not exist, the table
  may still be created. An error is not returned until the FTS4 table is
  read (if the uncompress function does not exist) or written (if it is the
  compress function that does not exist).

-- Create an FTS4 table that stores data in compressed form. This
-- assumes that the scalar functions zip() and unzip() have been (or
-- will be) added to the database handle.
CREATE VIRTUAL TABLE papers USING fts4(author, document, compress=zip, uncompress=unzip);



  When implementing the compress and uncompress functions it is important to
  pay attention to data types. Specifically, when a user reads a value from
  a compressed FTS table, the value returned by FTS is exactly the same
  as the value returned by the uncompress function, including the data type.
  If that data type is not the same as the data type of the original value as
  passed to the compress function (for example if the uncompress function is
  returning BLOB when compress was originally passed TEXT), then the users
  query may not function as expected.


c2: SQLite FTS3 and FTS4 Extensions
c3: 6.2. The content= option
c5: fts3.html#the_content_option_
c4:
The content option allows FTS4 to forego storing the text being indexed.
  The content option can be used in two ways:


 The indexed documents are not stored within the SQLite database
        at all (a "contentless" FTS4 table), or

 The indexed documents are stored in a database table created and
        managed by the user (an "external content" FTS4 table).



  Because the indexed documents themselves are usually much larger than
  the full-text index, the content option can be used to achieve
  significant space savings.


c2: SQLite FTS3 and FTS4 Extensions
c3: 6.2.1. Contentless FTS4 Tables
c5: fts3.html#_contentless_fts4_tables_
c4:
In order to create an FTS4 table that does not store a copy of the indexed
  documents at all, the content option should be set to an empty string.
  For example, the following SQL creates such an FTS4 table with three
  columns - "a", "b", and "c":

CREATE VIRTUAL TABLE t1 USING fts4(content="", a, b, c);



  Data can be inserted into such an FTS4 table using an INSERT statements.
  However, unlike ordinary FTS4 tables, the user must supply an explicit
  integer docid value. For example:

-- This statement is Ok:
INSERT INTO t1(docid, a, b, c) VALUES(1, 'a b c', 'd e f', 'g h i');

-- This statement causes an error, as no docid value has been provided:
INSERT INTO t1(a, b, c) VALUES('j k l', 'm n o', 'p q r');



  It is not possible to UPDATE or DELETE a row stored in a contentless FTS4
  table. Attempting to do so is an error.


  Contentless FTS4 tables also support SELECT statements. However, it is
  an error to attempt to retrieve the value of any table column other than
  the docid column. The auxiliary function matchinfo() may be used, but
  snippet() and offsets() may not. For example:

-- The following statements are Ok:
SELECT docid FROM t1 WHERE t1 MATCH 'xxx';
SELECT docid FROM t1 WHERE a MATCH 'xxx';
SELECT matchinfo(t1) FROM t1 WHERE t1 MATCH 'xxx';

-- The following statements all cause errors, as the value of columns
-- other than docid are required to evaluate them.
SELECT * FROM t1;
SELECT a, b FROM t1 WHERE t1 MATCH 'xxx';
SELECT docid FROM t1 WHERE a LIKE 'xxx%';
SELECT snippet(t1) FROM t1 WHERE t1 MATCH 'xxx';



  Errors related to attempting to retrieve column values other than docid
  are runtime errors that occur within sqlite3_step(). In some cases, for
  example if the MATCH expression in a SELECT query matches zero rows, there
  may be no error at all even if a statement does refer to column values
  other than docid.


c2: SQLite FTS3 and FTS4 Extensions
c3: 6.2.2. External Content FTS4 Tables
c5: fts3.html#_external_content_fts4_tables_
c4:
An "external content" FTS4 table is similar to a contentless table, except
  that if evaluation of a query requires the value of a column other than
  docid, FTS4 attempts to retrieve that value from a table (or view, or
  virtual table) nominated by the user (hereafter referred to as the "content
  table"). The FTS4 module never writes to the content table, and writing
  to the content table does not affect the full-text index. It is the
  responsibility of the user to ensure that the content table and the
  full-text index are consistent.


  An external content FTS4 table is created by setting the content option
  to the name of a table (or view, or virtual table) that may be queried by
  FTS4 to retrieve column values when required. If the nominated table does
  not exist, then an external content table behaves in the same way as
  a contentless table. For example:

CREATE TABLE t2(id INTEGER PRIMARY KEY, a, b, c);
CREATE VIRTUAL TABLE t3 USING fts4(content="t2", a, c);



  Assuming the nominated table does exist, then its columns must be the same
  as or a superset of those defined for the FTS table.  The external table
  must also be in the same database file as the FTS table.  In other words,
  The external table cannot be in a different database file connected using
  ATTACH nor may one of the FTS table and the external content be in the
  TEMP database when the other is in a persistent database file such as MAIN.


  When a users query on the FTS table requires a column value other than
  docid, FTS attempts to read the requested value from the corresponding column of
  the row in the content table with a rowid value equal to the current FTS
  docid. Only the subset of content-table columns duplicated in the FTS/34
  table declaration can be queried for - to retrieve values from any other
  columns the content table must be queried directly. Or, if such a row cannot
  be found in the content table, a NULL value is used instead. For example:

CREATE TABLE t2(id INTEGER PRIMARY KEY, a, b, c);
CREATE VIRTUAL TABLE t3 USING fts4(content="t2", b, c);

INSERT INTO t2 VALUES(2, 'a b', 'c d', 'e f');
INSERT INTO t2 VALUES(3, 'g h', 'i j', 'k l');
INSERT INTO t3(docid, b, c) SELECT id, b, c FROM t2;
-- The following query returns a single row with two columns containing
-- the text values "i j" and "k l".
--
-- The query uses the full-text index to discover that the MATCH
-- term matches the row with docid=3. It then retrieves the values
-- of columns b and c from the row with rowid=3 in the content table
-- to return.
--
SELECT * FROM t3 WHERE t3 MATCH 'k';

-- Following the UPDATE, the query still returns a single row, this
-- time containing the text values "xxx" and "yyy". This is because the
-- full-text index still indicates that the row with docid=3 matches
-- the FTS4 query 'k', even though the documents stored in the content
-- table have been modified.
--
UPDATE t2 SET b = 'xxx', c = 'yyy' WHERE rowid = 3;
SELECT * FROM t3 WHERE t3 MATCH 'k';

-- Following the DELETE below, the query returns one row containing two
-- NULL values. NULL values are returned because FTS is unable to find
-- a row with rowid=3 within the content table.
--
DELETE FROM t2;
SELECT * FROM t3 WHERE t3 MATCH 'k';



  When a row is deleted from an external content FTS4 table, FTS4 needs to
  retrieve the column values of the row being deleted from the content table.
  This is so that FTS4 can update the full-text index entries for each token
  that occurs within the deleted row to indicate that row has been
  deleted. If the content table row cannot be found, or if it contains values
  inconsistent with the contents of the FTS index, the results can be difficult
  to predict. The FTS index may be left containing entries corresponding to the
  deleted row, which can lead to seemingly nonsensical results being returned
  by subsequent SELECT queries. The same applies when a row is updated, as
  internally an UPDATE is the same as a DELETE followed by an INSERT.


  This means that in order to keep an FTS in sync with an external content
  table, any UPDATE or DELETE operations must be applied first to the FTS
  table, and then to the external content table. For example:

CREATE TABLE t1_real(id INTEGER PRIMARY KEY, a, b, c, d);
CREATE VIRTUAL TABLE t1_fts USING fts4(content="t1_real", b, c);

-- This works. When the row is removed from the FTS table, FTS retrieves
-- the row with rowid=123 and tokenizes it in order to determine the entries
-- that must be removed from the full-text index.
--
DELETE FROM t1_fts WHERE rowid = 123;
DELETE FROM t1_real WHERE rowid = 123;

-- This does not work. By the time the FTS table is updated, the row
-- has already been deleted from the underlying content table. As a result
-- FTS is unable to determine the entries to remove from the FTS index and
-- so the index and content table are left out of sync.
--
DELETE FROM t1_real WHERE rowid = 123;
DELETE FROM t1_fts WHERE rowid = 123;



  Instead of writing separately to the full-text index and the content table,
  some users may wish to use database triggers to keep the full-text index
  up to date with respect to the set of documents stored in the content table.
  For example, using the tables from earlier examples:

CREATE TRIGGER t2_bu BEFORE UPDATE ON t2 BEGIN
  DELETE FROM t3 WHERE docid=old.rowid;
END;
CREATE TRIGGER t2_bd BEFORE DELETE ON t2 BEGIN
  DELETE FROM t3 WHERE docid=old.rowid;
END;

CREATE TRIGGER t2_au AFTER UPDATE ON t2 BEGIN
  INSERT INTO t3(docid, b, c) VALUES(new.rowid, new.b, new.c);
END;
CREATE TRIGGER t2_ai AFTER INSERT ON t2 BEGIN
  INSERT INTO t3(docid, b, c) VALUES(new.rowid, new.b, new.c);
END;



  The DELETE trigger must be fired before the actual delete takes place
  on the content table. This is so that FTS4 can still retrieve the original
  values in order to update the full-text index. And the INSERT trigger must
  be fired after the new row is inserted, so as to handle the case where the
  rowid is assigned automatically within the system. The UPDATE trigger must
  be split into two parts, one fired before and one after the update of the
  content table, for the same reasons.


  The FTS4 "rebuild" command
  deletes the entire full-text index and rebuilds it based on the current
  set of documents in the content table. Assuming again that "t3" is the
  name of the external content FTS4 table, the rebuild command looks like this:

INSERT INTO t3(t3) VALUES('rebuild');



  This command may also be used with ordinary FTS4 tables, for example if
  the implementation of the tokenizer changes.  It is an
  error to attempt to rebuild the full-text index maintained by a contentless
  FTS4 table, since no content will be available to do the rebuilding.


c2: SQLite FTS3 and FTS4 Extensions
c3: 6.3. The languageid= option
c5: fts3.html#the_languageid_option
c4:
When the languageid option is present, it specifies the name of
  another hidden column that is added to the FTS4
  table and which is used to specify the language stored in each row
  of the FTS4 table.  The name of the languageid hidden column must
  be distinct from all other column names in the FTS4 table.  Example:

CREATE VIRTUAL TABLE t1 USING fts4(x, y, languageid="lid")



  The default value of a languageid column is 0. Any value inserted
  into a languageid column is converted to a 32-bit (not 64) signed
  integer.


  By default, FTS queries (those that use the MATCH operator)
  consider only those rows with the languageid column set to 0. To
  query for rows with other languageid values, a constraint of the
  form " = " must be added to the queries
  WHERE clause. For example:

SELECT * FROM t1 WHERE t1 MATCH 'abc' AND lid=5;



  It is not possible for a single FTS query to return rows with
  different languageid values. The results of adding WHERE clauses
  that use other operators (e.g.  lid!=5, or lid&lt;=5) are undefined.


  If the content option is used along with the languageid option,
  then the named languageid column must exist in the content= table
  (subject to the usual rules - if a query never needs to read the
  content table then this restriction does not apply).


  When the languageid option is used, SQLite invokes the xLanguageid()
  on the sqlite3_tokenizer_module object immediately after the object
  is created in order to pass in the language id that the
  tokenizer should use.  The xLanguageid() method will never be called
  more than once for any single tokenizer object.  The fact that different
  languages might be tokenized differently is one reason why no single
  FTS query can return rows with different languageid values.


c2: SQLite FTS3 and FTS4 Extensions
c3: 6.4. The matchinfo= option
c5: fts3.html#the_matchinfo_option
c4:
The matchinfo option may only be set to the value "fts3".
  Attempting to set matchinfo to anything other than "fts3" is an error.
  If this option is specified, then some of the extra information stored by
  FTS4 is omitted. This reduces the amount of disk space consumed by
  an FTS4 table until it is almost the same as the amount that would
  be used by the equivalent FTS3 table, but also means that the data
  accessed by passing the 'l' flag to the matchinfo() function is
  not available.


c2: SQLite FTS3 and FTS4 Extensions
c3: 6.5. The notindexed= option
c5: fts3.html#the_notindexed_option
c4:
Normally, the FTS module maintains an inverted index of all terms in
  all columns of the table. This option is used to specify the name of
  a column for which entries should not be added to the index. Multiple
  "notindexed" options may be used to specify that multiple columns should
  be omitted from the index. For example:

-- Create an FTS4 table for which only the contents of columns c2 and c4
-- are tokenized and added to the inverted index.
CREATE VIRTUAL TABLE t1 USING fts4(c1, c2, c3, c4, notindexed=c1, notindexed=c3);



  Values stored in unindexed columns are not eligible to match MATCH
  operators. They do not influence the results of the offsets() or matchinfo()
  auxiliary functions. Nor will the snippet() function ever return a
  snippet based on a value stored in an unindexed column.


c2: SQLite FTS3 and FTS4 Extensions
c3: 6.6. The prefix= option
c5: fts3.html#the_prefix_option
c4:
The FTS4 prefix option causes FTS to index term prefixes of specified lengths
  in the same way that it always indexes complete terms.  The prefix option
  must be set to a comma separated list of positive non-zero integers.
  For each value N in the list, prefixes of length N bytes (when encoded
  using UTF-8) are indexed.  FTS4 uses term prefix indexes to speed up
  prefix queries. The cost, of course, is that indexing term prefixes as
  well as complete terms increases the database size and slows down write
  operations on the FTS4 table.


  Prefix indexes may be used to optimize prefix queries in two cases.
  If the query is for a prefix of N bytes, then a prefix index created
  with "prefix=N" provides the best optimization. Or, if no "prefix=N"
  index is available, a "prefix=N+1" index may be used instead.
  Using a "prefix=N+1" index is less
  efficient than a "prefix=N" index, but is better than no prefix index at all.

-- Create an FTS4 table with indexes to optimize 2 and 4 byte prefix queries.
CREATE VIRTUAL TABLE t1 USING fts4(c1, c2, prefix="2,4");

-- The following two queries are both optimized using the prefix indexes.
SELECT * FROM t1 WHERE t1 MATCH 'ab*';
SELECT * FROM t1 WHERE t1 MATCH 'abcd*';

-- The following two queries are both partially optimized using the prefix
-- indexes. The optimization is not as pronounced as it is for the queries
-- above, but still an improvement over no prefix indexes at all.
SELECT * FROM t1 WHERE t1 MATCH 'a*';
SELECT * FROM t1 WHERE t1 MATCH 'abc*';


c2: SQLite FTS3 and FTS4 Extensions
c3: 7. Special Commands For FTS3 and FTS4
c5: fts3.html#commands
c4:
Special INSERT operates can be used to issue commands to FTS3 and FTS4 tables.
  Every FTS3 and FTS4 has a hidden, read-only column which is the same name as
  the table itself.  INSERTs into this hidden column are interpreted as commands
  to the FTS3/4 table.  For a table with the name "xyz" the following commands
  are supported:


INSERT INTO xyz(xyz) VALUES('optimize');
INSERT INTO xyz(xyz) VALUES('rebuild');
INSERT INTO xyz(xyz) VALUES('integrity-check');
INSERT INTO xyz(xyz) VALUES('merge=X,Y');
INSERT INTO xyz(xyz) VALUES('automerge=N');


c2: SQLite FTS3 and FTS4 Extensions
c3: 7.1. The "optimize" command
c5: fts3.html#optimize
c4:
The "optimize" command causes FTS3/4 to merge together all of its
  inverted index b-trees into one large and complete b-tree.  Doing
  an optimize will make subsequent queries run faster since there are
  fewer b-trees to search, and it may reduce disk usage by coalescing
  redundant entries.  However, for a large FTS table, running optimize
  can be as expensive as running VACUUM.  The optimize command
  essentially has to read and write the entire FTS table, resulting
  in a large transaction.


  In batch-mode operation, where an FTS table is initially built up
  using a large number of INSERT operations, then queried repeatedly
  without further changes, it is often a good idea
  to run "optimize" after the last INSERT and before the first query.


c2: SQLite FTS3 and FTS4 Extensions
c3: 7.2. The "rebuild" command
c5: fts3.html#rebuild
c4:
The "rebuild" command causes SQLite to discard the entire FTS3/4
  table and then rebuild it again from original text.  The concept
  is similar to REINDEX, only that it applies to an
  FTS3/4 table instead of an ordinary index.


  The "rebuild" command should be run whenever the implementation
  of a custom tokenizer changes, so that all content can be retokenized.
  The "rebuild" command is also useful when using the
  FTS4 content option after changes have been made to the original
  content table.


c2: SQLite FTS3 and FTS4 Extensions
c3: 7.3. The "integrity-check" command
c5: fts3.html#integcheck
c4:
The "integrity-check" command causes SQLite to read and verify
  the accuracy of all inverted indices in an FTS3/4 table by comparing
  those inverted indices against the original content. The
  "integrity-check" command silently succeeds if the inverted
  indices are all ok, but will fail with an SQLITE_CORRUPT error
  if any problems are found.


  The "integrity-check" command is similar in concept to
 PRAGMA integrity_check.  In a working system, the "integrity-command"
 should always be successful.  Possible causes of integrity-check
 failures include:
  
   The application has made changes to the FTS shadow tables
       directly, without using the FTS3/4 virtual table, causing
       the shadow tables to become out of sync with each other.
   Using the FTS4 content option and failing to manually keep
       the content in sync with the FTS4 inverted indices.
   Bugs in the FTS3/4 virtual table.  (The "integrity-check"
       command was original conceived as part of the test suite
       for FTS3/4.)
   Corruption to the underlying SQLite database file.  (See
       documentation on how to corrupt and SQLite database for
       additional information.)


c2: SQLite FTS3 and FTS4 Extensions
c3: 7.4. The "merge=X,Y" command
c5: fts3.html#mergecmd
c4:
The "merge=X,Y" command (where X and Y are integers) causes SQLite
  to do a limited amount of work toward merging the various inverted
  index b-trees of an FTS3/4 table together into one large b-tree.
  The X value is the target number of "blocks" to be merged, and Y is
  the minimum number of b-tree segments on a level required before
  merging will be applied to that level.  The value of Y should
  be between 2 and 16 with a recommended value of 8.  The value of X
  can be any positive integer but values on the order of 100 to 300
  are recommended.


  When an FTS table accumulates 16 b-tree segments at the same level,
  the next INSERT into that table will cause all 16 segments to be
  merged into a single b-tree segment at the next higher level.  The
  effect of these level merges is that most INSERTs into an FTS table
  are very fast and take minimal memory, but an occasional INSERT is
  slow and generates a large transaction because of the need to
  do merging. This results in "spiky" performance of INSERTs.


  To avoid spiky INSERT performance, an application can run the
  "merge=X,Y" command periodically, possibly in an idle thread or
  idle process, to ensure that the FTS table never accumulates
  too many b-tree segments at the same level.  INSERT performance
  spikes can generally be avoided, and performance of FTS3/4 can be
  maximized, by running "merge=X,Y" after every few thousand
  document inserts.  Each "merge=X,Y" command will run in a separate
  transaction (unless they are grouped together using BEGIN...COMMIT,
  of course).  The transactions can be kept small by choosing a value
  for X in the range of 100 to 300.  The idle thread that is running
  the merge commands can know when it is done by checking the difference
  in sqlite3_total_changes() before and after each "merge=X,Y"
  command and stopping the loop when the difference drops below two.


c2: SQLite FTS3 and FTS4 Extensions
c3: 7.5. The "automerge=N" command
c5: fts3.html#automerge
c4:
The "automerge=N" command (where N is an integer between 0 and 15,
  inclusive) is used to configure an FTS3/4 tables "automerge" parameter,
  which controls automatic incremental inverted index merging. The default
  automerge value for new tables is 0, meaning that automatic incremental
  merging is completely disabled. If the value of the automerge parameter
  is modified using the "automerge=N" command, the new parameter value is
  stored persistently in the database and is used by all subsequently
  established database connections.


  Setting the automerge parameter to a non-zero value enables automatic
  incremental merging. This causes SQLite to do a small amount of inverted
  index merging after every INSERT operation. The amount of merging
  performed is designed so that the FTS3/4 table never reaches a point
  where it has 16 segments at the same level and hence has to do a large
  merge in order to complete an insert.  In other words, automatic
  incremental merging is designed to prevent spiky INSERT performance.


  The downside of automatic incremental merging is that it makes
  every INSERT, UPDATE, and DELETE operation on an FTS3/4 table run
  a little slower, since extra time must be used to do the incremental
  merge.  For maximum performance, it is recommended that applications
  disable automatic incremental merge and instead use the
  "merge" command in an idle process to keep the inverted indices
  well merged.  But if the structure of an application does not easily
  allow for idle processes, the use of automatic incremental merge is
  a very reasonable fallback solution.


  The actual value of the automerge parameter determines the number of
  index segments merged simultaneously by an automatic inverted index
  merge. If the value is set to N, the system waits until there are at
  least N segments on a single level before beginning to incrementally
  merge them. Setting a lower value of N causes segments to be merged more
  quickly, which may speed up full-text queries and, if the workload
  contains UPDATE or DELETE operations as well as INSERTs, reduce the space
  on disk consumed by the full-text index. However, it also increases the
  amount of data written to disk.


  For general use in cases where the workload contains few UPDATE or DELETE
  operations, a good choice for automerge is 8. If the workload contains
  many UPDATE or DELETE commands,
  or if query speed is a concern, it may be advantageous to reduce automerge
  to 2.


  For reasons of backwards compatibility, the "automerge=1" command sets
  the automerge parameter to 8, not 1 (a value of 1 would make no sense
  anyway, as merging data from a single segment is a no-op).


c2: SQLite FTS3 and FTS4 Extensions
c3: 8. Tokenizers
c5: fts3.html#tokenizer
c4:
An FTS tokenizer is a set of rules for extracting terms from a document
  or basic FTS full-text query.


  Unless a specific tokenizer is specified as part of the CREATE
  VIRTUAL TABLE statement used to create the FTS table, the default
  tokenizer, "simple", is used. The simple tokenizer extracts tokens from
  a document or basic FTS full-text query according to the following
  rules:


   A term is a contiguous sequence of eligible characters, where
    eligible characters are all alphanumeric characters and all characters with
    Unicode codepoint values greater than or equal to 128.
    All other characters are
    discarded when splitting a document into terms. Their only contribution is
    to separate adjacent terms.

   All uppercase characters within the ASCII range (Unicode codepoints
    less than 128), are transformed to their lowercase equivalents as part
    of the tokenization process. Thus, full-text queries are
    case-insensitive when using the simple tokenizer.



  For example, when a document containing the text "Right now, they're very
  frustrated.", the terms extracted from the document and added to the
  full-text index are, in order, "right now they re very frustrated". Such
  a document would match a full-text query such as "MATCH 'Frustrated'",
  as the simple tokenizer transforms the term in the query to lowercase
  before searching the full-text index.


  As well as the "simple" tokenizer, the FTS source code features a tokenizer
  that uses the Porter
  Stemming algorithm. This tokenizer uses the same rules to separate
  the input document into terms including folding all terms into lower case,
  but also uses the Porter Stemming algorithm to reduce related English language
  words to a common root. For example, using the same input document as in the
  paragraph above, the porter tokenizer extracts the following tokens:
  "right now thei veri frustrat". Even though some of these terms are not even
  English words, in some cases using them to build the full-text index is more
  useful than the more intelligible output produced by the simple tokenizer.
  Using the porter tokenizer, the document not only matches full-text queries
  such as "MATCH 'Frustrated'", but also queries such as "MATCH 'Frustration'",
  as the term "Frustration" is reduced by the Porter stemmer algorithm to
  "frustrat" - just as "Frustrated" is. So, when using the porter tokenizer,
  FTS is able to find not just exact matches for queried terms, but matches
  against similar English language terms. For more information on the
  Porter Stemmer algorithm, please refer to the page linked above.


  Example illustrating the difference between the "simple" and "porter"
  tokenizers:

-- Create a table using the simple tokenizer. Insert a document into it.
CREATE VIRTUAL TABLE simple USING fts3(tokenize=simple);
INSERT INTO simple VALUES('Right now they''re very frustrated');

-- The first of the following two queries matches the document stored in
-- table "simple". The second does not.
SELECT * FROM simple WHERE simple MATCH 'Frustrated';
SELECT * FROM simple WHERE simple MATCH 'Frustration';

-- Create a table using the porter tokenizer. Insert the same document into it
CREATE VIRTUAL TABLE porter USING fts3(tokenize=porter);
INSERT INTO porter VALUES('Right now they''re very frustrated');

-- Both of the following queries match the document stored in table "porter".
SELECT * FROM porter WHERE porter MATCH 'Frustrated';
SELECT * FROM porter WHERE porter MATCH 'Frustration';



  If this extension is compiled with the SQLITE_ENABLE_ICU pre-processor
  symbol defined, then there exists a built-in tokenizer named "icu"
  implemented using the ICU library. The first argument passed to the
  xCreate() method (see fts3_tokenizer.h) of this tokenizer may be
  an ICU locale identifier. For example "tr_TR" for Turkish as used
  in Turkey, or "en_AU" for English as used in Australia. For example:

CREATE VIRTUAL TABLE thai_text USING fts3(text, tokenize=icu th_TH)



  The ICU tokenizer implementation is very simple. It splits the input
  text according to the ICU rules for finding word boundaries and discards
  any tokens that consist entirely of white-space. This may be suitable
  for some applications in some locales, but not all. If more complex
  processing is required, for example to implement stemming or
  discard punctuation, this can be done by creating a tokenizer
  implementation that uses the ICU tokenizer as part of its implementation.




  The "unicode61" tokenizer is available beginning with SQLite version 3.7.13
  (2012-06-11).
  Unicode61 works very much like "simple" except that it does simple unicode
  case folding according to rules in Unicode Version 6.1 and it recognizes
  unicode space and punctuation characters and uses those to separate tokens.
  The simple tokenizer only does case folding of ASCII characters and only
  recognizes ASCII space and punctuation characters as token separators.


  By default, "unicode61" attempts to remove diacritics from Latin script
  characters. This behaviour can be overridden by adding the tokenizer argument
  "remove_diacritics=0". For example:

-- Create tables that remove alldiacritics from Latin script characters
-- as part of tokenization.
CREATE VIRTUAL TABLE txt1 USING fts4(tokenize=unicode61);
CREATE VIRTUAL TABLE txt2 USING fts4(tokenize=unicode61 "remove_diacritics=2");

-- Create a table that does not remove diacritics from Latin script
-- characters as part of tokenization.
CREATE VIRTUAL TABLE txt3 USING fts4(tokenize=unicode61 "remove_diacritics=0");


The remove_diacritics option may be set to "0", "1" or "2". The default
   value is "1".  If it is set to "1" or "2", then diacritics are removed from
   Latin script characters as described above. However, if it is set to "1",
   then diacritics are not removed in the fairly uncommon case where a single
   unicode codepoint is used to represent a character with more that one
   diacritic. For example, diacritics are not removed from codepoint 0x1ED9
   ("LATIN SMALL LETTER O WITH CIRCUMFLEX AND DOT BELOW"). This is technically
   a bug, but cannot be fixed without creating backwards compatibility
   problems. If this option is set to "2", then diacritics are correctly
   removed from all Latin characters.


  It is also possible to customize the set of codepoints that unicode61 treats
  as separator characters. The "separators=" option may be used to specify one
  or more extra characters that should be treated as separator characters, and
  the "tokenchars=" option may be used to specify one or more extra characters
  that should be treated as part of tokens instead of as separator characters.
  For example:

-- Create a table that uses the unicode61 tokenizer, but considers "."
-- and "=" characters to be part of tokens, and capital "X" characters to
-- function as separators.
CREATE VIRTUAL TABLE txt3 USING fts4(tokenize=unicode61 "tokenchars=.=" "separators=X");

-- Create a table that considers space characters (codepoint 32) to be
-- a token character
CREATE VIRTUAL TABLE txt4 USING fts4(tokenize=unicode61 "tokenchars= ");



  If a character specified as part of the argument to "tokenchars=" is considered
  to be a token character by default, it is ignored. This is true even if it has
  been marked as a separator by an earlier "separators=" option. Similarly, if
  a character specified as part of a "separators=" option is treated as a separator
  character by default, it is ignored. If multiple "tokenchars=" or "separators="
  options are specified, all are processed. For example:

-- Create a table that uses the unicode61 tokenizer, but considers "."
-- and "=" characters to be part of tokens, and capital "X" characters to
-- function as separators. Both of the "tokenchars=" options are processed
-- The "separators=" option ignores the "." passed to it, as "." is by
-- default a separator character, even though it has been marked as a token
-- character by an earlier "tokenchars=" option.
CREATE VIRTUAL TABLE txt5 USING fts4(
    tokenize=unicode61 "tokenchars=." "separators=X." "tokenchars=="
);



  The arguments passed to the "tokenchars=" or "separators=" options are
  case-sensitive. In the example above, specifying that "X" is a separator
  character does not affect the way "x" is handled.


c2: SQLite FTS3 and FTS4 Extensions
c3: 8.1. Custom (Application Defined) Tokenizers
c5: fts3.html#custom_application_defined_tokenizers
c4:
In addition to providing built-in "simple", "porter" and (possibly) "icu" and
  "unicode61" tokenizers,
  FTS provides an interface for applications to implement and register custom
  tokenizers written in C.  The interface used to create a new tokenizer is defined
  and described in the fts3_tokenizer.h source file.


  Registering a new FTS tokenizer is similar to registering a new
  virtual table module with SQLite. The user passes a pointer to a
  structure containing pointers to various callback functions that
  make up the implementation of the new tokenizer type. For tokenizers,
  the structure (defined in fts3_tokenizer.h) is called
  "sqlite3_tokenizer_module".


  FTS does not expose a C-function that users call to register new
  tokenizer types with a database handle. Instead, the pointer must
  be encoded as an SQL blob value and passed to FTS through the SQL
  engine by evaluating a special scalar function, "fts3_tokenizer()".
  The fts3_tokenizer() function may be called with one or two arguments,
  as follows:

SELECT fts3_tokenizer(&lt;tokenizer-name&gt;);
SELECT fts3_tokenizer(&lt;tokenizer-name&gt;, &lt;sqlite3_tokenizer_module ptr&gt;);



  Where &lt;tokenizer-name&gt; is parameter to which a string is bound using
  sqlite3_bind_text() where the string identifies the tokenizer and
  &lt;sqlite3_tokenizer_module ptr&gt; is a parameter to which a BLOB is
  bound using sqlite3_bind_blob() where the value of the BLOB is a
  pointer to an sqlite3_tokenizer_module structure.
  If the second argument is present,
  it is registered as tokenizer &lt;tokenizer-name&gt; and a copy of it
  returned. If only one argument is passed, a pointer to the tokenizer
  implementation currently registered as &lt;tokenizer-name&gt; is returned,
  encoded as a blob. Or, if no such tokenizer exists, an SQL exception
  (error) is raised.


  Prior to SQLite version 3.11.0 (2016-02-15), the arguments to
  fts3_tokenizer() could be literal strings or BLOBs. They did not have to
  be bound parameters.  But that could lead to security problems in the
  event of an SQL injection.  Hence, the legacy behavior is now disabled
  by default.  But the old legacy behavior can be enabled, for backwards
  compatibility in applications that really need it,
  by calling
  sqlite3_db_config(db,SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER,1,0).


  The following block contains an example of calling the fts3_tokenizer()
  function from C code:

/*
** Register a tokenizer implementation with FTS3 or FTS4.
*/
int registerTokenizer(
  sqlite3 *db,
  char *zName,
  const sqlite3_tokenizer_module *p
){
  int rc;
  sqlite3_stmt *pStmt;
  const char *zSql = "SELECT fts3_tokenizer(?1, ?2)";

  rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
  if( rc!=SQLITE_OK ){
    return rc;
  }

  sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC);
  sqlite3_bind_blob(pStmt, 2, &p, sizeof(p), SQLITE_STATIC);
  sqlite3_step(pStmt);

  return sqlite3_finalize(pStmt);
}

/*
** Query FTS for the tokenizer implementation named zName.
*/
int queryTokenizer(
  sqlite3 *db,
  char *zName,
  const sqlite3_tokenizer_module **pp
){
  int rc;
  sqlite3_stmt *pStmt;
  const char *zSql = "SELECT fts3_tokenizer(?)";

  *pp = 0;
  rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
  if( rc!=SQLITE_OK ){
    return rc;
  }

  sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC);
  if( SQLITE_ROW==sqlite3_step(pStmt) ){
    if( sqlite3_column_type(pStmt, 0)==SQLITE_BLOB ){
      memcpy(pp, sqlite3_column_blob(pStmt, 0), sizeof(*pp));
    }
  }

  return sqlite3_finalize(pStmt);
}


c2: SQLite FTS3 and FTS4 Extensions
c3: 8.2. Querying Tokenizers
c5: fts3.html#querying_tokenizers
c4:
The "fts3tokenize" virtual table can be used to directly access any
   tokenizer.  The following SQL demonstrates how to create an instance
   of the fts3tokenize virtual table:

CREATE VIRTUAL TABLE tok1 USING fts3tokenize('porter');


The name of the desired tokenizer should be substituted in place of
   'porter' in the example, of course.  If the tokenizer requires one or
   more arguments, they should be separated by commas in the fts3tokenize
   declaration (even though they are separated by spaces in declarations
   of regular fts4 tables). The following creates fts4 and fts3tokenize
   tables that use the same tokenizer:
CREATE VIRTUAL TABLE text1 USING fts4(tokenize=icu en_AU);
CREATE VIRTUAL TABLE tokens1 USING fts3tokenize(icu, en_AU);

CREATE VIRTUAL TABLE text2 USING fts4(tokenize=unicode61 "tokenchars=@." "separators=123");
CREATE VIRTUAL TABLE tokens2 USING fts3tokenize(unicode61, "tokenchars=@.", "separators=123");



   Once the virtual table is created, it can be queried as follows:

SELECT token, start, end, position
  FROM tok1
 WHERE input='This is a test sentence.';


The virtual table will return one row of output for each token in the
   input string.  The "token" column is the text of the token.  The "start"
   and "end" columns are the byte offset to the beginning and end of the
   token in the original input string.
   The "position" column is the sequence number
   of the token in the original input string.  There is also an "input"
   column which is simply a copy of the input string that is specified in
   the WHERE clause.  Note that a constraint of the form "input=?" must
   appear in the WHERE clause or else the virtual table will have no input
   to tokenize and will return no rows.  The example above generates
   the following output:

thi|0|4|0
is|5|7|1
a|8|9|2
test|10|14|3
sentenc|15|23|4


Notice that the tokens in the result set from the fts3tokenize virtual
   table have been transformed according to the rules of the tokenizer.
   Since this example used the "porter" tokenizer, the "This" token was
   converted into "thi".  If the original text of the token is desired,
   it can be retrieved using the "start" and "end" columns with the
   substr() function.  For example:

SELECT substr(input, start+1, end-start), token, position
  FROM tok1
 WHERE input='This is a test sentence.';


The fts3tokenize virtual table can be used on any tokenizer, regardless
   of whether or not there exists an FTS3 or FTS4 table that actually uses
   that tokenizer.


c2: SQLite FTS3 and FTS4 Extensions
c3: 9. Data Structures
c5: fts3.html#data_structures
c4:
This section describes at a high-level the way the FTS module stores its
  index and content in the database. It is not necessary to read or
  understand the material in this section in order to use FTS in an
  application. However, it may be useful to application developers attempting
  to analyze and understand FTS performance characteristics, or to developers
  contemplating enhancements to the existing FTS feature set.


c2: SQLite FTS3 and FTS4 Extensions
c3: 9.1. Shadow Tables
c5: fts3.html#shadow_tables
c4:
For each FTS virtual table in a database, three to five real (non-virtual) tables
  are created to store the underlying data.  These real tables are called "shadow tables".
  The real tables are named "%_content",
  "%_segdir", "%_segments", "%_stat", and "%_docsize", where "%" is replaced by the name
  of the FTS virtual table.


  The leftmost column of the "%_content" table is an INTEGER PRIMARY KEY field
  named "docid". Following this is one column for each column of the FTS
  virtual table as declared by the user, named by prepending the column name
  supplied by the user with "cN", where N is the index of the
  column within the table, numbered from left to right starting with 0. Data
  types supplied as part of the virtual table declaration are not used as
  part of the %_content table declaration. For example:

-- Virtual table declaration
CREATE VIRTUAL TABLE abc USING fts4(a NUMBER, b TEXT, c);

-- Corresponding %_content table declaration
CREATE TABLE abc_content(docid INTEGER PRIMARY KEY, c0a, c1b, c2c);



  The %_content table contains the unadulterated data inserted by the user
  into the FTS virtual table by the user. If the user does not explicitly
  supply a "docid" value when inserting records, one is selected automatically
  by the system.


  The %_stat and %_docsize tables are only created if the FTS table uses the
  FTS4 module, not FTS3. Furthermore, the %_docsize table is omitted if the
  FTS4 table is created with the "matchinfo=fts3" directive
  specified as part of the CREATE VIRTUAL TABLE statement. If they are created,
  the schema of the two tables is as follows:
CREATE TABLE %_stat(
  id INTEGER PRIMARY KEY,
  value BLOB
);

CREATE TABLE %_docsize(
  docid INTEGER PRIMARY KEY,
  size BLOB
);



  For each row in the FTS table, the %_docsize table contains a corresponding
  row with the same "docid" value. The "size" field contains a blob consisting
  of N FTS varints, where N is the number of user-defined columns
  in the table. Each varint in the "size" blob is the number of tokens in the
  corresponding column of the associated row in the FTS table. The %_stat table
  always contains a single row with the "id" column set to 0. The "value"
  column contains a blob consisting of N+1 FTS varints, where N
  is again the number of user-defined columns in the FTS table. The first
  varint in the blob is set to the total number of rows in the FTS table. The
  second and subsequent varints contain the total number of tokens stored in
  the corresponding column for all rows of the FTS table.


  The two remaining tables, %_segments and %_segdir, are used to store the
  full-text index. Conceptually, this index is a lookup table that maps each
  term (word) to the set of docid values corresponding to records in the
  %_content table that contain one or more occurrences of the term. To
  retrieve all documents that contain a specified term, the FTS module
  queries this index to determine the set of docid values for records that
  contain the term, then retrieves the required documents from the %_content
  table. Regardless of the schema of the FTS virtual table, the %_segments
  and %_segdir tables are always created as follows:

CREATE TABLE %_segments(
  blockid INTEGER PRIMARY KEY,       -- B-tree node id
  block blob                         -- B-tree node data
);

CREATE TABLE %_segdir(
  level INTEGER,
  idx INTEGER,
  start_block INTEGER,               -- Blockid of first node in %_segments
  leaves_end_block INTEGER,          -- Blockid of last leaf node in %_segments
  end_block INTEGER,                 -- Blockid of last node in %_segments
  root BLOB,                         -- B-tree root node
  PRIMARY KEY(level, idx)
);



  The schema depicted above is not designed to store the full-text index
  directly. Instead, it is used to store one or more b-tree structures. There
  is one b-tree for each row in the %_segdir table. The %_segdir table
  row contains the root node and various meta-data associated with the
  b-tree structure, and the %_segments table contains all other (non-root)
  b-tree nodes. Each b-tree is referred to as a "segment". Once it has
  been created, a segment b-tree is never updated (although it may be
  deleted altogether).


  The keys used by each segment b-tree are terms (words). As well as the
  key, each segment b-tree entry has an associated "doclist" (document list).
  A doclist consists of zero or more entries, where each entry consists of:


   A docid (document id), and
   A list of term offsets, one for each occurrence of the term within
       the document. A term offset indicates the number of tokens (words)
       that occur before the term in question, not the number of characters
       or bytes. For example, the term offset of the term "war" in the
       phrase "Ancestral voices prophesying war!" is 3.



  Entries within a doclist are sorted by docid. Positions within a doclist
  entry are stored in ascending order.


  The contents of the logical full-text index is found by merging the
  contents of all segment b-trees. If a term is present in more than one
  segment b-tree, then it maps to the union of each individual doclist. If,
  for a single term, the same docid occurs in more than one doclist, then only
  the doclist that is part of the most recently created segment b-tree is
  considered valid.


  Multiple b-tree structures are used instead of a single b-tree to reduce
  the cost of inserting records into FTS tables. When a new record is
  inserted into an FTS table that already contains a lot of data, it is
  likely that many of the terms in the new record are already present in
  a large number of existing records. If a single b-tree were used, then
  large doclist structures would have to be loaded from the database,
  amended to include the new docid and term-offset list, then written back
  to the database. Using multiple b-tree tables allows this to be avoided
  by creating a new b-tree which can be merged with the existing b-tree
  (or b-trees) later on. Merging of b-tree structures can be performed as
  a background task, or once a certain number of separate b-tree structures
  have been accumulated. Of course, this scheme makes queries more expensive
  (as the FTS code may have to look up individual terms in more than one
  b-tree and merge the results), but it has been found that in practice this
  overhead is often negligible.


c2: SQLite FTS3 and FTS4 Extensions
c3: 9.2. Variable Length Integer (varint) Format
c5: fts3.html#variable_length_integer_varint_format
c4:
Integer values stored as part of segment b-tree nodes are encoded using the
  FTS varint format. This encoding is similar, but not identical, to
  the SQLite varint format.


  An encoded FTS varint consumes between one and ten bytes of space. The
  number of bytes required is determined by the sign and magnitude of the
  integer value encoded. More accurately, the number of bytes used to store
  the encoded integer depends on the position of the most significant set bit
  in the 64-bit twos-complement representation of the integer value. Negative
  values always have the most significant bit set (the sign bit), and so are
  always stored using the full ten bytes. Positive integer values may be
  stored using less space.


  The final byte of an encoded FTS varint has its most significant bit
  cleared. All preceding bytes have the most significant bit set. Data
  is stored in the remaining seven least significant bits of each byte.
  The first byte of the encoded representation contains the least significant
  seven bits of the encoded integer value. The second byte of the encoded
  representation, if it is present, contains the seven next least significant
  bits of the integer value, and so on. The following table contains examples
  of encoded integer values:


  DecimalHexadecimalEncoded Representation
  430x000000000000002B0x2B
  2008150x000000000003106F0xEF 0xA0 0x0C
  -10xFFFFFFFFFFFFFFFF0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0x01


c2: SQLite FTS3 and FTS4 Extensions
c3: 9.3. Segment B-Tree Format
c5: fts3.html#segment_b_tree_format
c4:
Segment b-trees are prefix-compressed b+-trees. There is one segment b-tree
  for each row in the %_segdir table (see above). The root node of the segment
  b-tree is stored as a blob in the "root" field of the corresponding row
  of the %_segdir table. All other nodes (if any exist) are stored in the
  "blob" column of the %_segments table. Nodes within the %_segments table are
  identified by the integer value in the blockid field of the corresponding
  row. The following table describes the fields of the %_segdir table:


  Column           Interpretation
  level            
    Between them, the contents of the "level" and "idx" fields define the
    relative age of the segment b-tree. The smaller the value stored in the
    "level" field, the more recently the segment b-tree was created. If two
    segment b-trees are of the same "level", the segment with the larger
    value stored in the "idx" column is more recent. The PRIMARY KEY constraint
    on the %_segdir table prevents any two segments from having the same value
    for both the "level" and "idx" fields.
  idx               See above.
  start_block      
    The blockid that corresponds to the node with the smallest blockid that
    belongs to this segment b-tree. Or zero if the entire segment b-tree
    fits on the root node. If it exists, this node is always a leaf node.
  leaves_end_block 
    The blockid that corresponds to the leaf node with the largest blockid
    that belongs to this segment b-tree. Or zero if the entire segment b-tree
    fits on the root node.
  end_block 
    This field may contain either an integer or a text field consisting of
    two integers separated by a space character (unicode codepoint 0x20).

    The first, or only, integer is the blockid that corresponds to the interior
    node with the largest blockid that belongs to this segment b-tree. Or zero
    if the entire segment b-tree fits on the root node. If it exists, this node
    is always an interior node.

    The second integer, if it is present, is the aggregate size of all data
    stored on leaf pages in bytes. If the value is negative, then the segment
    is the output of an unfinished incremental-merge operation, and the
    absolute value is current size in bytes.

  root             
    Blob containing the root node of the segment b-tree.



  Apart from the root node, the nodes that make up a single segment b-tree are
  always stored using a contiguous sequence of blockids. Furthermore, the
  nodes that make up a single level of the b-tree are themselves stored as
  a contiguous block, in b-tree order. The contiguous sequence of blockids
  used to store the b-tree leaves are allocated starting with the blockid
  value stored in the "start_block" column of the corresponding %_segdir row,
  and finishing at the blockid value stored in the "leaves_end_block"
  field of the same row. It is therefore possible to iterate through all the
  leaves of a segment b-tree, in key order, by traversing the %_segments
  table in blockid order from "start_block" to "leaves_end_block".


c2: SQLite FTS3 and FTS4 Extensions
c3: 9.3.1. Segment B-Tree Leaf Nodes
c5: fts3.html#segment_b_tree_leaf_nodes
c4:
The following diagram depicts the format of a segment b-tree leaf node.


  
   Segment B-Tree Leaf Node Format



  The first term stored on each node ("Term 1" in the figure above) is
  stored verbatim. Each subsequent term is prefix-compressed with respect
  to its predecessor. Terms are stored within a page in sorted (memcmp)
  order.


c2: SQLite FTS3 and FTS4 Extensions
c3: 9.3.2. Segment B-Tree Interior Nodes
c5: fts3.html#segment_b_tree_interior_nodes
c4:
The following diagram depicts the format of a segment b-tree interior
  (non-leaf) node.


  
   Segment B-Tree Interior Node Format


c2: SQLite FTS3 and FTS4 Extensions
c3: 9.4. Doclist Format
c5: fts3.html#doclist_format
c4:
A doclist consists of an array of 64-bit signed integers, serialized using
  the FTS varint format. Each doclist entry is made up of a series of two
  or more integers, as follows:


   The docid value. The first entry in a doclist contains the literal docid
       value. The first field of each subsequent doclist entry contains the
       difference between the new docid and the previous one (always a positive
       number).
   Zero or more term-offset lists. A term-offset list is present for each
       column of the FTS virtual table that contains the term. A term-offset
       list consists of the following:
     
        Constant value 1. This field is omitted for any term-offset list
            associated with column 0.
        The column number (1 for the second leftmost column, etc.). This
            field is omitted for any term-offset list associated with column 0.
        A list of term-offsets, sorted from smallest to largest. Instead
            of storing the term-offset value literally, each integer stored
            is the difference between the current term-offset and the previous
            one (or zero if the current term-offset is the first), plus 2.
     
   Constant value 0.



  
   FTS3 Doclist Format



  
   FTS Doclist Entry Format



  For doclists for which the term appears in more than one column of the FTS
  virtual table, term-offset lists within the doclist are stored in column
  number order. This ensures that the term-offset list associated with
  column 0 (if any) is always first, allowing the first two fields of the
  term-offset list to be omitted in this case.


c2: SQLite FTS3 and FTS4 Extensions
c3: 10.1. UTF-16 byte-order-mark problem
c5: fts3.html#_utf_16_byte_order_mark_problem_
c4:
For UTF-16 databases, when using the "simple" tokenizer, it is possible to use
malformed unicode strings to cause the
integrity-check special command to falsely report
corruption, or for auxiliary functions to return
incorrect results. More specifically, the bug can be triggered by any of the following:


  A UTF-16 byte-order-mark (BOM) is embedded at the beginning of an SQL string
       literal value inserted into an FTS3 table. For example:

INSERT INTO fts_table(col) VALUES(char(0xfeff)||'text...');


  Malformed UTF-8 that SQLite converts to a UTF-16 byte-order-mark is
       embedded at the beginning of an SQL string literal value inserted
       into an FTS3 table.

  A text value created by casting a blob that begins with the two
       bytes 0xFF and 0xFE, in either possible order, is inserted into an
       FTS3 table. For example:

INSERT INTO fts_table(col) VALUES(CAST(X'FEFF' AS TEXT));



Everything works correctly if any of the following are true:

 The database encoding is UTF-8.
 All text strings are insert using one of the
     sqlite3_bind_text() family of functions.
 Literal strings contain no byte-order-marks.
 A tokenizer is used that recognizes byte-order-marks
     as whitespace.  (The default "simple" tokenizer for
     FTS3/4 does not think that BOMs are whitespace, but
     the unicode tokenizer does.)

All of the above conditions must be false in order for problems
to occur.  And even if all of the conditiona above are false,
most things will still operator correctly.  Only the
integrity-check command and the
auxiliary functions might given
unexpected results.


c2: SQLite FTS3 and FTS4 Extensions
c3: Appendix A: Search Application Tips
c5: fts3.html#appendix_a
c4:
FTS is primarily designed to support Boolean full-text queries - queries
  to find the set of documents that match a specified criteria. However, many
  (most?) search applications require that results are somehow ranked in order
  of "relevance", where "relevance" is defined as the likelihood that the user
  who performed the search is interested in a specific element of the returned
  set of documents. When using a search engine to find documents on the world
  wide web, the user expects that the most useful, or "relevant", documents
  will be returned as the first page of results, and that each subsequent page
  contains progressively less relevant results. Exactly how a machine can
  determine document relevance based on a users query is a complicated problem
  and the subject of much ongoing research.


  One very simple scheme might be to count the number of instances of the
  users search terms in each result document. Those documents that contain
  many instances of the terms are considered more relevant than those with
  a small number of instances of each term. In an FTS application, the
  number of term instances in each result could be determined by counting
  the number of integers in the return value of the offsets function.
  The following example shows a query that could be used to obtain the
  ten most relevant results for a query entered by the user:

-- This example (and all others in this section) assumes the following schema
CREATE VIRTUAL TABLE documents USING fts3(title, content);

-- Assuming the application has supplied an SQLite user function named "countintegers"
-- that returns the number of space-separated integers contained in its only argument,
-- the following query could be used to return the titles of the 10 documents that contain
-- the greatest number of instances of the users query terms. Hopefully, these 10
-- documents will be those that the users considers more or less the most "relevant".
SELECT title FROM documents
  WHERE documents MATCH &lt;query&gt;
  ORDER BY countintegers(offsets(documents)) DESC
  LIMIT 10 OFFSET 0



  The query above could be made to run faster by using the FTS matchinfo
  function to determine the number of query term instances that appear in each
  result. The matchinfo function is much more efficient than the offsets
  function. Furthermore, the matchinfo function provides extra information
  regarding the overall number of occurrences of each query term in the entire
  document set (not just the current row) and the number of documents in which
  each query term appears. This may be used (for example) to attach a higher
  weight to less common terms which may increase the overall computed relevancy
  of those results the user considers more interesting.

-- If the application supplies an SQLite user function called "rank" that
-- interprets the blob of data returned by matchinfo and returns a numeric
-- relevancy based on it, then the following SQL may be used to return the
-- titles of the 10 most relevant documents in the dataset for a users query.
SELECT title FROM documents
  WHERE documents MATCH &lt;query&gt;
  ORDER BY rank(matchinfo(documents)) DESC
  LIMIT 10 OFFSET 0



  The SQL query in the example above uses less CPU than the first example
  in this section, but still has a non-obvious performance problem. SQLite
  satisfies this query by retrieving the value of the "title" column and
  matchinfo data from the FTS module for every row matched by the users
  query before it sorts and limits the results. Because of the way SQLite's
  virtual table interface works, retrieving the value of the "title" column
  requires loading the entire row from disk (including the "content" field,
  which may be quite large). This means that if the users query matches
  several thousand documents, many megabytes of "title" and "content" data
  may be loaded from disk into memory even though they will never be used
  for any purpose.


  The SQL query in the following example block is one solution to this
  problem. In SQLite, when a sub-query
  used in a join contains a LIMIT clause, the results of the sub-query are
  calculated and stored in temporary table before the main query is executed.
  This means that SQLite will load only the docid and matchinfo data for each
  row matching the users query into memory, determine the docid values
  corresponding to the ten most relevant documents, then load only the title
  and content information for those 10 documents only. Because both the matchinfo
  and docid values are gleaned entirely from the full-text index, this results
  in dramatically less data being loaded from the database into memory.

SELECT title FROM documents JOIN (
    SELECT docid, rank(matchinfo(documents)) AS rank
    FROM documents
    WHERE documents MATCH &lt;query&gt;
    ORDER BY rank DESC
    LIMIT 10 OFFSET 0
) AS ranktable USING(docid)
ORDER BY ranktable.rank DESC



  The next block of SQL enhances the query with solutions to two other problems
  that may arise in developing search applications using FTS:


   
       The snippet function cannot be used with the above query. Because
       the outer query does not include a "WHERE ... MATCH" clause, the snippet
       function may not be used with it. One solution is to duplicate the WHERE
       clause used by the sub-query in the outer query. The overhead associated
       with this is usually negligible.
   
       The relevancy of a document may depend on something other than just
       the data available in the return value of matchinfo. For example
       each document in the database may be assigned a static weight based
       on factors unrelated to its content (origin, author, age, number
       of references etc.). These values can be stored by the application
       in a separate table that can be joined against the documents table
       in the sub-query so that the rank function may access them.



  This version of the query is very similar to that used by the
  sqlite.org documentation search
  application.

-- This table stores the static weight assigned to each document in FTS table
-- "documents". For each row in the documents table there is a corresponding row
-- with the same docid value in this table.
CREATE TABLE documents_data(docid INTEGER PRIMARY KEY, weight);

-- This query is similar to the one in the block above, except that:
--
--   1. It returns a "snippet" of text along with the document title for display. So
--      that the snippet function may be used, the "WHERE ... MATCH ..." clause from
--      the sub-query is duplicated in the outer query.
--
--   2. The sub-query joins the documents table with the document_data table, so that
--      implementation of the rank function has access to the static weight assigned
--      to each document.
SELECT title, snippet(documents) FROM documents JOIN (
    SELECT docid, rank(matchinfo(documents), documents_data.weight) AS rank
    FROM documents JOIN documents_data USING(docid)
    WHERE documents MATCH &lt;query&gt;
    ORDER BY rank DESC
    LIMIT 10 OFFSET 0
) AS ranktable USING(docid)
WHERE documents MATCH &lt;query&gt;
ORDER BY ranktable.rank DESC



  All the example queries above return the ten most relevant query results.
  By modifying the values used with the OFFSET and LIMIT clauses, a query
  to return (say) the next ten most relevant results is easy to construct.
  This may be used to obtain the data required for a search applications second
  and subsequent pages of results.


  The next block contains an example rank function that uses matchinfo data
  implemented in C. Instead of a single weight, it allows a weight to be
  externally assigned to each column of each document. It may be registered
  with SQLite like any other user function using sqlite3_create_function.

 Security Warning: Because it is just an ordinary SQL function,
  rank() may be invoked as part of any SQL query in any context. This means
  that the first argument passed may not be a valid matchinfo blob.
  Implementors should take care to handle this case without causing buffer
  overruns or other potential security problems.

/*
** SQLite user defined function to use with matchinfo() to calculate the
** relevancy of an FTS match. The value returned is the relevancy score
** (a real value greater than or equal to zero). A larger value indicates 
** a more relevant document.
**
** The overall relevancy returned is the sum of the relevancies of each 
** column value in the FTS table. The relevancy of a column value is the
** sum of the following for each reportable phrase in the FTS query:
**
**   (&lt;hit count&gt; / &lt;global hit count&gt;) * &lt;column weight&gt;
**
** where &lt;hit count&gt; is the number of instances of the phrase in the
** column value of the current row and &lt;global hit count&gt; is the number
** of instances of the phrase in the same column of all rows in the FTS
** table. The &lt;column weight&gt; is a weighting factor assigned to each
** column by the caller (see below).
**
** The first argument to this function must be the return value of the FTS 
** matchinfo() function. Following this must be one argument for each column 
** of the FTS table containing a numeric weight factor for the corresponding 
** column. Example:
**
**     CREATE VIRTUAL TABLE documents USING fts3(title, content)
**
** The following query returns the docids of documents that match the full-text
** query &lt;query&gt; sorted from most to least relevant. When calculating
** relevance, query term instances in the 'title' column are given twice the
** weighting of those in the 'content' column.
**
**     SELECT docid FROM documents 
**     WHERE documents MATCH &lt;query&gt; 
**     ORDER BY rank(matchinfo(documents), 1.0, 0.5) DESC
*/
static void rankfunc(sqlite3_context *pCtx, int nVal, sqlite3_value **apVal){
  int *aMatchinfo;                /* Return value of matchinfo() */
  int nMatchinfo;                 /* Number of elements in aMatchinfo&#x5B;&#x5D; */
  int nCol = 0;                   /* Number of columns in the table */
  int nPhrase = 0;                /* Number of phrases in the query */
  int iPhrase;                    /* Current phrase */
  double score = 0.0;             /* Value to return */

  assert( sizeof(int)==4 );

  /* Check that the number of arguments passed to this function is correct.
  ** If not, jump to wrong_number_args. Set aMatchinfo to point to the array
  ** of unsigned integer values returned by FTS function matchinfo. Set
  ** nPhrase to contain the number of reportable phrases in the users full-text
  ** query, and nCol to the number of columns in the table. Then check that the
  ** size of the matchinfo blob is as expected. Return an error if it is not.
  */
  if( nVal&lt;1 ) goto wrong_number_args;
  aMatchinfo = (unsigned int *)sqlite3_value_blob(apVal&#x5B;0&#x5D;);
  nMatchinfo = sqlite3_value_bytes(apVal&#x5B;0&#x5D;) / sizeof(int);
  if( nMatchinfo&gt;=2 ){
    nPhrase = aMatchinfo&#x5B;0&#x5D;;
    nCol = aMatchinfo&#x5B;1&#x5D;;
  }
  if( nMatchinfo!=(2+3*nCol*nPhrase) ){
    sqlite3_result_error(pCtx,
      "invalid matchinfo blob passed to function rank()", -1);
    return;
  }
  if( nVal!=(1+nCol) ) goto wrong_number_args;

  /* Iterate through each phrase in the users query. */
  for(iPhrase=0; iPhrase&lt;nPhrase; iPhrase++){
    int iCol;                     /* Current column */

    /* Now iterate through each column in the users query. For each column,
    ** increment the relevancy score by:
    **
    **   (&lt;hit count&gt; / &lt;global hit count&gt;) * &lt;column weight&gt;
    **
    ** aPhraseinfo&#x5B;&#x5D; points to the start of the data for phrase iPhrase. So
    ** the hit count and global hit counts for each column are found in 
    ** aPhraseinfo&#x5B;iCol*3&#x5D; and aPhraseinfo&#x5B;iCol*3+1&#x5D;, respectively.
    */
    int *aPhraseinfo = &aMatchinfo&#x5B;2 + iPhrase*nCol*3&#x5D;;
    for(iCol=0; iCol&lt;nCol; iCol++){
      int nHitCount = aPhraseinfo&#x5B;3*iCol&#x5D;;
      int nGlobalHitCount = aPhraseinfo&#x5B;3*iCol+1&#x5D;;
      double weight = sqlite3_value_double(apVal&#x5B;iCol+1&#x5D;);
      if( nHitCount>0 ){
        score += ((double)nHitCount / (double)nGlobalHitCount) * weight;
      }
    }
  }

  sqlite3_result_double(pCtx, score);
  return;

  /* Jump here if the wrong number of arguments are passed to this function */
wrong_number_args:
  sqlite3_result_error(pCtx, "wrong number of arguments to function rank()", -1);
}


c2: JSON Functions And Operators
c3: 1. Overview
c5: json1.html#overview
c4:
By default, SQLite supports sixteen functions and two operators for
dealing with JSON values.  There are also two table-valued functions
that can be used to decompose a JSON string.


There are 15 scalar functions and operators:



json(json)



json_array(value1,value2,...)



json_array_length(json)json_array_length(json,path)



json_error_position(json)



json_extract(json,path,...)



json -> path



json ->> path



json_insert(json,path,value,...)



json_object(label1,value1,...)



json_patch(json1,json2)



json_remove(json,path,...)



json_replace(json,path,value,...)



json_set(json,path,value,...)



json_type(json)json_type(json,path)



json_valid(json)



json_quote(value)





There are two aggregate SQL functions:



json_group_array(value)



json_group_object(label,value)





The two table-valued functions are:



json_each(json)json_each(json,path)



json_tree(json)json_tree(json,path)


c2: JSON Functions And Operators
c3: 2. Compiling in JSON Support
c5: json1.html#compiling_in_json_support
c4:
The JSON functions and operators are built into SQLite by default,
as of SQLite version 3.38.0 (2022-02-22).  They can be omitted
by adding the -DSQLITE_OMIT_JSON compile-time option.  Prior to
version 3.38.0, the JSON functions were an extension that would only
be included in builds if the -DSQLITE_ENABLE_JSON1 compile-time option
was included.  In other words, the JSON functions went from being
opt-in with SQLite version 3.37.2 and earlier to opt-out with
SQLite version 3.38.0 and later.


c2: JSON Functions And Operators
c3: 3. Interface Overview
c5: json1.html#interface_overview
c4:
SQLite stores JSON as ordinary text.
Backwards compatibility constraints mean that SQLite is only able to
store values that are NULL, integers, floating-point numbers, text,
and BLOBs.  It is not possible to add a sixth "JSON" type.


SQLite does not (currently) support a binary encoding
of JSON.  Experiments have been unable to find a binary encoding
that is smaller or faster than a plain text encoding.
(The present implementation parses JSON text at over 250 MB/s.)
All JSON functions currently throw an error if any of their
arguments are BLOBs because BLOBs are reserved
for a future enhancement in which BLOBs will store the binary encoding
for JSON.


c2: JSON Functions And Operators
c3: 3.1. JSON arguments
c5: json1.html#json_arguments
c4:
For functions that accept JSON as their first argument, that argument
can be a JSON object, array, number, string, or null.  SQLite numeric
values and NULL values are interpreted as JSON numbers and nulls, respectively.
SQLite text values can be understood as JSON objects, arrays, or strings.
If an SQLite text value that is not a well-formed JSON object, array, or
string is passed into JSON function, that function will usually throw
an error.  (Exceptions to this rule are json_valid(),
json_quote(), and json_error_position().)


These routines understand all
rfc-7159 JSON syntax
and also JSON5 extensions.  JSON text
generated by these routines always strictly conforms to the
canonical JSON definition and does not contain any JSON5
or other extensions.  The ability to read and understand JSON5 was added in
version 3.42.0 (2023-05-16).
Prior versions of SQLite would only read canonical JSON.


c2: JSON Functions And Operators
c3: 3.2. PATH arguments
c5: json1.html#path_arguments
c4:
For functions that accept PATH arguments, that PATH must be well-formed or
else the function will throw an error.
A well-formed PATH is a text value that begins with exactly one
'$' character followed by zero or more instances
of ".objectlabel" or "&#91;arrayindex&#93;".


The arrayindex is usually a non-negative integer N.  In
that case, the array element selected is the N-th element
of the array, starting with zero on the left.
The arrayindex can also be of the form "#-N"
in which case the element selected is the N-th from the
right.  The last element of the array is "#-1".  Think of
the "#" characters as the "number of elements in the array".  Then
the expression "#-1" evaluates to the integer that corresponds to 
the last entry in the array.  It is sometimes useful for the array
index to be just the # character, for example when appending
a value to an existing JSON array:


json_set('[0,1,2]','$[#]','new')
&rarr; '[0,1,2,"new"]'


c2: JSON Functions And Operators
c3: 3.3. VALUE arguments
c5: json1.html#value_arguments
c4:
For functions that accept "value" arguments (also shown as
"value1" and "value2"),
those arguments are usually understood
to be literal strings that are quoted and become JSON string values
in the result.  Even if the input value strings look like 
well-formed JSON, they are still interpreted as literal strings in the
result.


However, if a value argument comes directly from the result of another
JSON function or from the -> operator (but not the ->> operator),
then the argument is understood to be actual JSON and
the complete JSON is inserted rather than a quoted string.


For example, in the following call to json_object(), the value
argument looks like a well-formed JSON array.  However, because it is just
ordinary SQL text, it is interpreted as a literal string and added to the
result as a quoted string:


json_object('ex','[52,3.14159]')
&rarr; '{"ex":"[52,3.14159]"}'

json_object('ex',('52,3.14159]'->>'$'))
&rarr; '{"ex":"[52,3.14159]"}'





But if the value argument in the outer json_object() call is the
result of another JSON function like json() or json_array(), then
the value is understood to be actual JSON and is inserted as such:


json_object('ex',json('[52,3.14159]'))
&rarr; '{"ex":[52,3.14159]}'

json_object('ex',json_array(52,3.14159))
&rarr; '{"ex":[52,3.14159]}'

json_object('ex','[52,3.14159]'->'$')
&rarr; '{"ex":[52,3.14159]}'





To be clear: "json" arguments are always interpreted as JSON
regardless of where the value for that argument comes from.  But
"value" arguments are only interpreted as JSON if those arguments
come directly from another JSON function or the -> operator.


Within JSON value arguments interpreted as JSON strings, Unicode escape
sequences are not treated as equivalent to the characters or escaped
control characters represented by the expressed Unicode code point.
Such escape sequences are not translated or specially treated; they
are treated as plain text by SQLite's JSON functions.


c2: JSON Functions And Operators
c3: 3.4. Compatibility
c5: json1.html#compatibility
c4:
The current implementation of this JSON library uses a recursive descent
parser.  In order to avoid using excess stack space, any JSON input that has
more than 1000 levels of nesting is considered invalid.   Limits on nesting
depth are allowed for compatible implementations of JSON by
RFC-7159 section 9.


c2: JSON Functions And Operators
c3: 3.5. JSON5 Extensions
c5: json1.html#json5_extensions
c4:
Beginning in version 3.42.0 (2023-05-16), these routines will
read and interpret input JSON text that includes
JSON5 extensions.  However, JSON text generated
by these routines will always be strictly conforming to the 
canonical definition of JSON.


Here is a synopsis of JSON5 extensions (adapted from the
JSON5 specification):


 Object keys may be unquoted identifiers.
 Objects may have a single trailing comma.
 Arrays may have a single trailing comma.
 Strings may be single quoted.
 Strings may span multiple lines by escaping new line characters.
 Strings may include new character escapes.
 Numbers may be hexadecimal.
 Numbers may have a leading or trailing decimal point.
 Numbers may be "Infinity", "-Infinity", and "NaN".
 Numbers may begin with an explicit plus sign.
 Single (//...) and multi-line (/*...*/) comments are allowed.
 Additional white space characters are allowed.



To convert string X from JSON5 into canonical JSON, invoke
"json(X)".  The output of the "json()" function will be canonical
JSON regardless of any JSON5 extensions that are present in the input.
For backwards compatibility, the json_valid(X) function continues
to report false for inputs that are not canonical JSON, even if the
input is JSON5 that the function is able to understand.  To determine
whether or not an input string is valid JSON5,
use the expression: "json_error_position(X)==0".


These routines understand all of JSON5, plus a little more.
SQLite extends the JSON5 syntax in these two ways:



Strict JSON5 requires that
unquoted object keys must be ECMAScript 5.1 IdentifierNames.  But large
unicode tables and lots of code is required in order to determine whether or
not a key is an ECMAScript 5.1 IdentifierName.  For this reason,
SQLite allows object keys to include any unicode characters
greater than U+007f that are not whitespace characters.  This relaxed
definition of "identifier" greatly simplifies the implementation and allows
the JSON parser to be smaller and run faster.


JSON5 allows floating-point infinities to be expressed as
"Infinity", "-Infinity", or "+Infinity"
in exactly that case - the initial "I" is capitalized and all other
characters are lower case.  SQLite also allows the abbreviation "Inf"
to be used in place of "Infinity" and it allows both keywords
to appear in any combination of upper and lower case letters.
Similarly,
JSON5 allows "NaN" for not-a-number.  SQLite extends this to also allow
"QNaN" and "SNaN" in any combination of upper and lower case letters.
Note that SQLite interprets NaN, QNaN, and SNaN as just an alternative
spellings for "null".
This extension has been added because (we are told) there exists a lot
of JSON in the wild that includes these non-standard representations
for infinity and not-a-number.


c2: JSON Functions And Operators
c3: 4. Function Details
c5: json1.html#function_details
c4:
The following sections provide additional detail on the operation of
the various JSON functions and operators:


c2: JSON Functions And Operators
c3: 4.1. The json() function
c5: json1.html#the_json_function
c4:
The json(X) function verifies that its argument X is a valid
JSON string and returns a minified version of that JSON string
(with all unnecessary whitespace removed).  If X is not a well-formed
JSON string, then this routine throws an error.

In other words, this function converts raw text that looks like
JSON into actual JSON so that it may be passed into the value argument
of some other json function and will be interpreted as JSON rather than
a string.  This function is not appropriate for testing whether or not
a particular string is well-formed JSON - use the json_valid() and/or
json_error_position() routines below for that task.

If the argument X to json(X) contains JSON objects with duplicate
labels, then it is undefined whether or not the duplicates are
preserved.  The current implementation preserves duplicates.
However, future enhancements
to this routine may choose to silently remove duplicates.


Example:


json(' { "this" : "is", "a": [ "test" ] } ')
&rarr; '{"this":"is","a":["test"]}'


c2: JSON Functions And Operators
c3: 4.2. The json_array() function
c5: json1.html#the_json_array_function
c4:
The json_array() SQL function accepts zero or more arguments and
returns a well-formed JSON array that is composed from those arguments.
If any argument to json_array() is a BLOB then an error is thrown.

An argument with SQL type TEXT is normally converted into a quoted 
JSON string.  However, if the argument is the output from another json1
function, then it is stored as JSON.  This allows calls to json_array()
and json_object() to be nested.  The json() function can also
be used to force strings to be recognized as JSON.

Examples:


json_array(1,2,'3',4)
&rarr; '[1,2,"3",4]'

json_array('[1,2]')
&rarr; '["[1,2]"]'

json_array(json_array(1,2))
&rarr; '[[1,2]]'

json_array(1,null,'3','[4,5]','{"six":7.7}')
&rarr; '[1,null,"3","[4,5]","{\"six\":7.7}"]'

json_array(1,null,'3',json('[4,5]'),json('{"six":7.7}'))
&rarr; '[1,null,"3",[4,5],{"six":7.7}]'


c2: JSON Functions And Operators
c3: 4.3. The json_array_length() function
c5: json1.html#the_json_array_length_function
c4:
The json_array_length(X) function returns the number of elements
in the JSON array X, or 0 if X is some kind of JSON value other
than an array.  The json_array_length(X,P) locates the array at path P
within X and returns the length of that array, or 0 if path P locates
an element in X that is not a JSON array, and NULL if path P does not
locate any element of X.  Errors are thrown if either X is not 
well-formed JSON or if P is not a well-formed path.

Examples:


json_array_length('[1,2,3,4]')
&rarr; 4

json_array_length('[1,2,3,4]', '$')
&rarr; 4

json_array_length('[1,2,3,4]', '$[2]')
&rarr; 0

json_array_length('{"one":[1,2,3]}')
&rarr; 0

json_array_length('{"one":[1,2,3]}', '$.one')
&rarr; 3

json_array_length('{"one":[1,2,3]}', '$.two')
&rarr; NULL


c2: JSON Functions And Operators
c3: 4.4. The json_error_position() function
c5: json1.html#the_json_error_position_function
c4:
The json_error_positionf(X) function returns 0 if the input X is a
well-formed JSON or JSON5 string.  If the input X contains one or more
syntax errors, then this function returns the character position of the
first syntax error.  The left-most character is position 1.

This routine is useful for at least two purposes:



 To determine is a text string X is valid JSON or JSON5 as understood
    by SQLite, run "json_error_position(X)==0".  This is similar
    to json_valid() except that json_valid(X) requires X to be strictly
    conforming canonical JSON whereas json_error_position() allows the
    input to contains JSON5 extensions.

 Use this routine to find the location of a syntax error in a large
    JSON string during interactive debugging, or to generate a better
    error messages for human users.



The json_error_position() function was added with
SQLite version 3.42.0 (2023-05-16).


c2: JSON Functions And Operators
c3: 4.5. The json_extract() function
c5: json1.html#the_json_extract_function
c4:
The json_extract(X,P1,P2,...) extracts and returns one or more 
values from the
well-formed JSON at X.  If only a single path P1 is provided, then the
SQL datatype of the result is NULL for a JSON null, INTEGER or REAL
for a JSON numeric value, an INTEGER zero for a JSON false value,
an INTEGER one for a JSON true value, the dequoted text for a 
JSON string value, and a text representation for JSON object and array values.
If there are multiple path arguments (P1, P2, and so forth) then this
routine returns SQLite text which is a well-formed JSON array holding
the various values.

Examples:


json_extract('{"a":2,"c":[4,5,{"f":7}]}', '$')
&rarr; '{"a":2,"c":[4,5,{"f":7}]}'

json_extract('{"a":2,"c":[4,5,{"f":7}]}', '$.c')
&rarr; '[4,5,{"f":7}]'

json_extract('{"a":2,"c":[4,5,{"f":7}]}', '$.c[2]')
&rarr; '{"f":7}'

json_extract('{"a":2,"c":[4,5,{"f":7}]}', '$.c[2].f')
&rarr; 7

json_extract('{"a":2,"c":[4,5],"f":7}','$.c','$.a')
&rarr; '[[4,5],2]'

json_extract('{"a":2,"c":[4,5],"f":7}','$.c[#-1]')
&rarr; 5

json_extract('{"a":2,"c":[4,5,{"f":7}]}', '$.x')
&rarr; NULL

json_extract('{"a":2,"c":[4,5,{"f":7}]}', '$.x', '$.a')
&rarr; '[null,2]'

json_extract('{"a":"xyz"}', '$.a')
&rarr; 'xyz'

json_extract('{"a":null}', '$.a')
&rarr; NULL




There is a subtle incompatibility between the json_extract() function
in SQLite and the json_extract() function in MySQL.  The MySQL version
of json_extract() always returns JSON.  The SQLite version of
json_extract() only returns JSON if there are two or more PATH arguments
(because the result is then a JSON array) or if the single PATH argument
references an array or object.  In SQLite, if json_extract() has only
a single PATH argument and that PATH references a JSON null or a string
or a numeric value, then json_extract() returns the corresponding SQL
NULL, TEXT, INTEGER, or REAL value.

The difference between MySQL json_extract() and SQLite json_extract()
really only stands out when accessing individual values within the JSON
that are strings or NULLs.  The following table demonstrates the difference:



OperationSQLite ResultMySQL Result
json_extract('{"a":null,"b":"xyz"}','$.a')NULL'null'
json_extract('{"a":null,"b":"xyz"}','$.b')'xyz''"xyz"'


c2: JSON Functions And Operators
c3: 4.6. The -> and ->> operators
c5: json1.html#the_and_operators
c4:
Beginning with SQLite version 3.38.0 (2022-02-22), the ->
and ->> operators are available for extracting subcomponents of JSON.
The SQLite implementation of -> and ->> strives to be
compatible with both MySQL and PostgreSQL.
The -> and ->> operators take a JSON string
as their left operand and a PATH expression or object field
label or array index as their right operand.  The -> operator
returns a JSON representation of the selected subcomponent or NULL if that
subcomponent does not exist.  The ->> operator returns an SQL TEXT,
INTEGER, REAL, or NULL value that represents the selected subcomponent,
or NULL if the subcomponent does not exist.

Both the -> and ->> operators select the same subcomponent
of the JSON to their left.  The difference is that -> always returns a
JSON representation of that subcomponent and the ->> operator always
returns an SQL representation of that subcomponent.  Thus, these operators
are subtly different from a two-argument json_extract() function call.
A call to json_extract() with two arguments will return a JSON representation
of the subcomponent if and only if the subcomponent is a JSON array or
object, and will return an SQL representation of the subcomponent if the
subcomponent is a JSON null, string, or numeric value.

The right-hand operand to the -> and ->> operators can
be a well-formed JSON path expression.  This is the form used by MySQL.
For compatibility with PostgreSQL,
the -> and ->> operators also accept a text label or
integer as their right-hand operand.  If the right operand is a text
label X, then it is interpreted as the JSON path '$.X'.  If the right
operand is an integer value N, then it is interpreted as the JSON path '$[N]'.

Examples:


'{"a":2,"c":[4,5,{"f":7}]}' -> '$'
&rarr; '{"a":2,"c":[4,5,{"f":7}]}'

'{"a":2,"c":[4,5,{"f":7}]}' -> '$.c'
&rarr; '[4,5,{"f":7}]'

'{"a":2,"c":[4,5,{"f":7}]}' -> 'c'
&rarr; '[4,5,{"f":7}]'

'{"a":2,"c":[4,5,{"f":7}]}' -> '$.c[2]'
&rarr; '{"f":7}'

'{"a":2,"c":[4,5,{"f":7}]}' -> '$.c[2].f'
&rarr; '7'

'{"a":2,"c":[4,5,{"f":7}]}' ->> '$.c[2].f'
&rarr; 7

'{"a":2,"c":[4,5,{"f":7}]}' -> 'c' -> 2 ->> 'f'
&rarr; 7

'{"a":2,"c":[4,5],"f":7}' -> '$.c[#-1]'
&rarr; '5'

'{"a":2,"c":[4,5,{"f":7}]}' -> '$.x'
&rarr; NULL

'[11,22,33,44]' -> 3
&rarr; '44'

'[11,22,33,44]' ->> 3
&rarr; 44

'{"a":"xyz"}' -> '$.a'
&rarr; '"xyz"'

'{"a":"xyz"}' ->> '$.a'
&rarr; 'xyz'

'{"a":null}' -> '$.a'
&rarr; 'null'

'{"a":null}' ->> '$.a'
&rarr; NULL


c2: JSON Functions And Operators
c3: 4.7. The json_insert(), json_replace, and json_set() functions
c5: json1.html#the_json_insert_json_replace_and_json_set_functions
c4:
The json_insert(), json_replace, and json_set() functions all take
a single JSON value as their first argument followed by zero or more
pairs of path and value arguments, and return a new JSON string formed
by updating the input JSON by the path/value pairs.  The functions
differ only in how they deal with creating new values and overwriting
preexisting values.




FunctionOverwrite if already exists?Create if does not exist?

json_insert()NoYes

json_replace()YesNo

json_set()YesYes


The json_insert(), json_replace(), and json_set() functions always
take an odd number of arguments.  The first argument is always the original
JSON to be edited.  Subsequent arguments occur in pairs with the first
element of each pair being a path and the second element being the value
to insert or replace or set on that path.

Edits occur sequentially from left to right.  Changes caused by
prior edits can affect the path search for subsequent edits.

If the value of a path/value pair is an SQLite TEXT value, then it
is normally inserted as a quoted JSON string, even if the string looks
like valid JSON.  However, if the value is the result of another
json function (such as json() or json_array() or json_object())
or if it is the result of the -> operator,
then it is interpreted as JSON and is inserted as JSON retaining all
of its substructure.  Values that are the result of the ->> operator
are always interpreted as TEXT and are inserted as a JSON string even
if they look like valid JSON.

These routines throw an error if the first JSON argument is not
well-formed or if any PATH argument is not well-formed or if any
argument is a BLOB.

To append an element onto the end of an array, using json_insert()
with an array index of "#".  Examples:


json_insert('[1,2,3,4]','$[#]',99)
&rarr; '[1,2,3,4,99]'

json_insert('[1,[2,3],4]','$[1][#]',99)
&rarr; '[1,[2,3,99],4]'




Other examples:


json_insert('{"a":2,"c":4}', '$.a', 99)
&rarr; '{"a":2,"c":4}'

json_insert('{"a":2,"c":4}', '$.e', 99)
&rarr; '{"a":2,"c":4,"e":99}'

json_replace('{"a":2,"c":4}', '$.a', 99)
&rarr; '{"a":99,"c":4}'

json_replace('{"a":2,"c":4}', '$.e', 99)
&rarr; '{"a":2,"c":4}'

json_set('{"a":2,"c":4}', '$.a', 99)
&rarr; '{"a":99,"c":4}'

json_set('{"a":2,"c":4}', '$.e', 99)
&rarr; '{"a":2,"c":4,"e":99}'

json_set('{"a":2,"c":4}', '$.c', '[97,96]')
&rarr; '{"a":2,"c":"[97,96]"}'

json_set('{"a":2,"c":4}', '$.c', json('[97,96]'))
&rarr; '{"a":2,"c":[97,96]}'

json_set('{"a":2,"c":4}', '$.c', json_array(97,96))
&rarr; '{"a":2,"c":[97,96]}'


c2: JSON Functions And Operators
c3: 4.8. The json_object() function
c5: json1.html#the_json_object_function
c4:
The json_object() SQL function accepts zero or more pairs of arguments
and returns a well-formed JSON object that is composed from those arguments.
The first argument of each pair is the label and the second argument of
each pair is the value.
If any argument to json_object() is a BLOB then an error is thrown.

The json_object() function currently allows duplicate labels without
complaint, though this might change in a future enhancement.

An argument with SQL type TEXT it is normally converted into a quoted 
JSON string even if the input text is well-formed JSON.  
However, if the argument is the direct result from another JSON
function or the -> operator (but not the ->> operator), 
then it is treated as JSON and all of its JSON type information
and substructure is preserved.  This allows calls to json_object()
and json_array() to be nested.  The json() function can also
be used to force strings to be recognized as JSON.

Examples:


json_object('a',2,'c',4)
&rarr; '{"a":2,"c":4}'

json_object('a',2,'c','{e:5}')
&rarr; '{"a":2,"c":"{e:5}"}'

json_object('a',2,'c',json_object('e',5))
&rarr; '{"a":2,"c":{"e":5}}'


c2: JSON Functions And Operators
c3: 4.9. The json_patch() function
c5: json1.html#the_json_patch_function
c4:
The json_patch(T,P) SQL function runs the
RFC-7396 MergePatch algorithm
to apply patch P against input T.  The patched copy of T is returned.

MergePatch can add, modify, or delete elements of a JSON Object,
and so for JSON Objects, the json_patch() routine is a generalized
replacement for json_set() and json_remove().  However, MergePatch
treats JSON Array objects as atomic.  MergePatch cannot append to an
Array nor modify individual elements of an Array.  It can only insert,
replace, or delete the whole Array as a single unit.  Hence, json_patch()
is not as useful when dealing with JSON that includes Arrays,
especially Arrays with lots of substructure.

Examples:


json_patch('{"a":1,"b":2}','{"c":3,"d":4}')
&rarr; '{"a":1,"b":2,"c":3,"d":4}'

json_patch('{"a":[1,2],"b":2}','{"a":9}')
&rarr; '{"a":9,"b":2}'

json_patch('{"a":[1,2],"b":2}','{"a":null}')
&rarr; '{"b":2}'

json_patch('{"a":1,"b":2}','{"a":9,"b":null,"c":8}')
&rarr; '{"a":9,"c":8}'

json_patch('{"a":{"x":1,"y":2},"b":3}','{"a":{"y":9},"c":8}')
&rarr; '{"a":{"x":1,"y":9},"b":3,"c":8}'


c2: JSON Functions And Operators
c3: 4.10. The json_remove() function
c5: json1.html#the_json_remove_function
c4:
The json_remove(X,P,...) function takes a single JSON value as its
first argument followed by zero or more path arguments.
The json_remove(X,P,...) function returns
a copy of the X parameter with all the elements 
identified by path arguments removed.  Paths that select elements
not found in X are silently ignored.

Removals occurs sequentially from left to right.  Changes caused by
prior removals can affect the path search for subsequent arguments.

If the json_remove(X) function is called with no path arguments,
then it returns the input X reformatted, with excess whitespace
removed.

The json_remove() function throws an error if the first argument
is not well-formed JSON or if any later argument is not a well-formed
path, or if any argument is a BLOB.

Examples:


json_remove('[0,1,2,3,4]','$[2]')
&rarr; '[0,1,3,4]'

json_remove('[0,1,2,3,4]','$[2]','$[0]')
&rarr; '[1,3,4]'

json_remove('[0,1,2,3,4]','$[0]','$[2]')
&rarr; '[1,2,4]'

json_remove('[0,1,2,3,4]','$[#-1]','$[0]')
&rarr; '[1,2,3]'

json_remove('{"x":25,"y":42}')
&rarr; '{"x":25,"y":42}'

json_remove('{"x":25,"y":42}','$.z')
&rarr; '{"x":25,"y":42}'

json_remove('{"x":25,"y":42}','$.y')
&rarr; '{"x":25}'

json_remove('{"x":25,"y":42}','$')
&rarr; NULL


c2: JSON Functions And Operators
c3: 4.11. The json_type() function
c5: json1.html#the_json_type_function
c4:
The json_type(X) function returns the "type" of the outermost element
of X.  The json_type(X,P) function returns the "type" of the element
in X that is selected by path P.  The "type" returned by json_type() is
one of the following SQL text values:
'null', 'true', 'false', 'integer', 'real', 'text', 'array', or 'object'.
If the path P in json_type(X,P) selects an element that does not exist
in X, then this function returns NULL.

The json_type() function throws an error if any of its arguments is
not well-formed or is a BLOB.

Examples:


json_type('{"a":[2,3.5,true,false,null,"x"]}')
&rarr; 'object'

json_type('{"a":[2,3.5,true,false,null,"x"]}','$')
&rarr; 'object'

json_type('{"a":[2,3.5,true,false,null,"x"]}','$.a')
&rarr; 'array'

json_type('{"a":[2,3.5,true,false,null,"x"]}','$.a[0]')
&rarr; 'integer'

json_type('{"a":[2,3.5,true,false,null,"x"]}','$.a[1]')
&rarr; 'real'

json_type('{"a":[2,3.5,true,false,null,"x"]}','$.a[2]')
&rarr; 'true'

json_type('{"a":[2,3.5,true,false,null,"x"]}','$.a[3]')
&rarr; 'false'

json_type('{"a":[2,3.5,true,false,null,"x"]}','$.a[4]')
&rarr; 'null'

json_type('{"a":[2,3.5,true,false,null,"x"]}','$.a[5]')
&rarr; 'text'

json_type('{"a":[2,3.5,true,false,null,"x"]}','$.a[6]')
&rarr; NULL


c2: JSON Functions And Operators
c3: 4.12. The json_valid() function
c5: json1.html#the_json_valid_function
c4:
The json_valid(X) function return 1 if the argument X is well-formed
canonical RFC-7159 JSON without any extensions, or return 0 if the
argument X is not well-formed JSON or is JSON that includes
JSON5 extensions.

Examples:


json_valid('{"x":35}')
&rarr; 1

json_valid('{"x":35')
&rarr; 0

json_valid(NULL)
&rarr; NULL




Use the expression "json_error_position(X)==0" to determine if
a string is well-formed JSON5.  Use the "json(X)" routine to convert
JSON5 into canonical JSON.


c2: JSON Functions And Operators
c3: 4.13. The json_quote() function
c5: json1.html#the_json_quote_function
c4:
The json_quote(X) function converts the SQL value X (a number or a
string) into its corresponding JSON representation.  If X is a JSON value
returned by another JSON function, then this function is a no-op.

Examples:


json_quote(3.14159)
&rarr; 3.14159

json_quote('verdant')
&rarr; '"verdant"'

json_quote('[1]')
&rarr; '"[1]"'

json_quote(json('[1]'))
&rarr; '[1]'

json_quote('[1,')
&rarr; '"[1,"'


c2: JSON Functions And Operators
c3: 4.14. The json_group_array() and json_group_object() aggregate SQL functions
c5: json1.html#the_json_group_array_and_json_group_object_aggregate_sql_functions
c4:
The json_group_array(X) function is an
aggregate SQL function that returns a JSON array
comprised of all X values in the aggregation.
Similarly, the json_group_object(NAME,VALUE) function returns a JSON object
comprised of all NAME/VALUE pairs in the aggregation.


c2: JSON Functions And Operators
c3: 4.15. The json_each() and json_tree() table-valued functions
c5: json1.html#the_json_each_and_json_tree_table_valued_functions
c4:
The json_each(X) and json_tree(X) table-valued functions walk the
JSON value provided as their first argument and return one row for each
element.  The json_each(X) function only walks the immediate children
of the top-level array or object,
or just the top-level element itself if the top-level
element is a primitive value.
The json_tree(X) function recursively walks through the
JSON substructure starting with the top-level element.  

The json_each(X,P) and json_tree(X,P) functions work just like
their one-argument counterparts except that they treat the element
identified by path P as the top-level element.

The schema for the table returned by json_each() and json_tree() is
as follows:


CREATE TABLE json_tree(
    key ANY,             -- key for current element relative to its parent
    value ANY,           -- value for the current element
    type TEXT,           -- 'object','array','string','integer', etc.
    atom ANY,            -- value for primitive types, null for array & object
    id INTEGER,          -- integer ID for this element
    parent INTEGER,      -- integer ID for the parent of this element
    fullkey TEXT,        -- full path describing the current element
    path TEXT,           -- path to the container of the current row
    json JSON HIDDEN,    -- 1st input parameter: the raw JSON
    root TEXT HIDDEN     -- 2nd input parameter: the PATH at which to start
);



The "key" column is the integer array index for elements of a JSON array 
and the text label for elements of a JSON object.  The key column is
NULL in all other cases.


The "atom" column is the SQL value corresponding to primitive elements - 
elements other than JSON arrays and objects.  The "atom" column is NULL
for a JSON array or object.  The "value" column is the same as the
"atom" column for primitive JSON elements but takes on the text JSON value
for arrays and objects.


The "type" column is an SQL text value taken from ('null', 'true', 'false',
'integer', 'real', 'text', 'array', 'object') according to the type of
the current JSON element.


The "id" column is an integer that identifies a specific JSON element
within the complete JSON string.  The "id" integer is an internal housekeeping
number, the computation of which might change in future releases.  The
only guarantee is that the "id" column will be different for every row.


The "parent" column is always NULL for json_each().
For json_tree(),
the "parent" column is the "id" integer for the parent of the current
element, or NULL for the top-level JSON element or the element identified
by the root path in the second argument.


The "fullkey" column is a text path that uniquely identifies the current
row element within the original JSON string.  The complete key to the
true top-level element is returned even if an alternative starting point
is provided by the "root" argument.


The "path" column is the path to the array or object container that holds 
the current row, or the path to the current row in the case where the 
iteration starts on a primitive type and thus only provides a single
row of output.


c2: JSON Functions And Operators
c3: 4.15.1. Examples using json_each() and json_tree()
c5: json1.html#examples_using_json_each_and_json_tree_
c4:
Suppose the table "CREATE TABLE user(name,phone)" stores zero or
more phone numbers as a JSON array object in the user.phone field.
To find all users who have any phone number with a 704 area code:


SELECT DISTINCT user.name
  FROM user, json_each(user.phone)
 WHERE json_each.value LIKE '704-%';


Now suppose the user.phone field contains plain text if the user
has only a single phone number and a JSON array if the user has multiple
phone numbers.  The same question is posed: "Which users have a phone number
in the 704 area code?"  But now the json_each() function can only be called
for those users that have two or more phone numbers since json_each()
requires well-formed JSON as its first argument:


SELECT name FROM user WHERE phone LIKE '704-%'
UNION
SELECT user.name
  FROM user, json_each(user.phone)
 WHERE json_valid(user.phone)
   AND json_each.value LIKE '704-%';


Consider a different database with "CREATE TABLE big(json JSON)".
To see a complete line-by-line decomposition of the data:


SELECT big.rowid, fullkey, value
  FROM big, json_tree(big.json)
 WHERE json_tree.type NOT IN ('object','array');


In the previous, the "type NOT IN ('object','array')" term of the
WHERE clause suppresses containers and only lets through leaf elements.
The same effect could be achieved this way:


SELECT big.rowid, fullkey, atom
  FROM big, json_tree(big.json)
 WHERE atom IS NOT NULL;


Suppose each entry in the BIG table is a JSON object 
with a '$.id' field that is a unique identifier
and a '$.partlist' field that can be a deeply nested object.
You want to find the id of every entry that contains one
or more references to uuid '6fa5181e-5721-11e5-a04e-57f3d7b32808' anywhere
in its '$.partlist'.


SELECT DISTINCT json_extract(big.json,'$.id')
  FROM big, json_tree(big.json, '$.partlist')
 WHERE json_tree.key='uuid'
   AND json_tree.value='6fa5181e-5721-11e5-a04e-57f3d7b32808';

This page last modified on  2023-10-24 18:03:24 UTC


c2: Partial Indexes
c3: 1. Introduction
c5: partialindex.html#introduction
c4:
A partial index is an index over a subset of the rows of a table.



In ordinary indexes, there is exactly one entry in the index for every
row in the table.  In partial indexes, only some subset of the rows in the
table have corresponding index entries.  For example, a partial index might
omit entries for which the column being indexed is NULL.  When used 
judiciously, partial indexes can result in smaller database files and
improvements in both query and write performance.


c2: Partial Indexes
c3: 2. Creating Partial Indexes
c5: partialindex.html#creating_partial_indexes
c4:
Create a partial index by adding a WHERE clause to the end of an 
ordinary CREATE INDEX statement.


create-index-stmt:
hide
 
 
expr:
show
 
 
filter-clause:
show
 
 

function-arguments:
show
 
 
ordering-term:
show
 
 


literal-value:
show
 
 

over-clause:
show
 
 
frame-spec:
show
 
 

ordering-term:
show
 
 


raise-function:
show
 
 

select-stmt:
show
 
 
common-table-expression:
show
 
 

compound-operator:
show
 
 

join-clause:
show
 
 
join-constraint:
show
 
 

join-operator:
show
 
 


ordering-term:
show
 
 

result-column:
show
 
 

table-or-subquery:
show
 
 

window-defn:
show
 
 
frame-spec:
show
 
 



type-name:
show
 
 
signed-number:
show
 
 



indexed-column:
show
 
 





Any index that includes the WHERE clause at the end is considered to be
a partial index.  Indexes that omit the WHERE clause (or indexes that
are created by UNIQUE or PRIMARY KEY constraints inside of CREATE TABLE
statements) are ordinary full indexes.



The expression following the WHERE clause may contain operators,
literal values, and names of columns in the table being indexed.
The WHERE clause may not contain subqueries, references to other
tables, non-deterministic functions, or bound parameters.


Only rows of the table for which the WHERE clause evaluates to true
are included in the index.  If the WHERE clause expression evaluates 
to NULL or to false for some rows of the table, then those rows are omitted 
from the index.



The columns referenced in the WHERE clause of a partial index can be
any of the columns in the table, not just columns that happen to be
indexed.  However, it is very common for the WHERE clause
expression of a partial index to be a simple expression on the column
being indexed.  The following is a typical example:

CREATE INDEX po_parent ON purchaseorder(parent_po) WHERE parent_po IS NOT NULL;


In the example above, if most purchase orders do not have a "parent"
purchase order, then most parent_po values will be NULL.  That means only
a small subset of the rows in the purchaseorder table will be indexed.
Hence the index will take up much less space.  And changes to the original
purchaseorder table will run faster since the po_parent index only needs
to be updated for those exceptional rows where parent_po is not NULL.
But the index is still useful for querying.  In particular, if one wants
to know all "children" of a particular purchase order "?1", the query
would be:

SELECT po_num FROM purchaseorder WHERE parent_po=?1;


The query above will use the po_parent index to help find the answer,
since the po_parent index contains entries for all rows of interest.
Note that since po_parent is smaller than a full index, the query will
likely run faster too.


c2: Partial Indexes
c3: 2.1. Unique Partial Indexes
c5: partialindex.html#unique_partial_indexes
c4:
A partial index definition may include the UNIQUE keyword.  If it
does, then SQLite requires every entry in the index to be unique.
This provides a mechanism for enforcing uniqueness across some subset of
the rows in a table.

For example, suppose you have a database of the members of a large
organization where each person is assigned to a particular "team".  
Each team has a "leader" who is also a member of that team.  The
table might look something like this:

CREATE TABLE person(
  person_id       INTEGER PRIMARY KEY,
  team_id         INTEGER REFERENCES team,
  is_team_leader  BOOLEAN,
  -- other fields elided
);


The team_id field cannot be unique because there are usually multiple people
on the same team.  One cannot make the combination of team_id and is_team_leader
unique since there are usually multiple non-leaders on each team.  The
solution to enforcing one leader per team is to create a unique index
on team_id but restricted to those entries for which is_team_leader is
true:

CREATE UNIQUE INDEX team_leader ON person(team_id) WHERE is_team_leader;


Coincidentally, that same index is useful for locating the team leader
of a particular team:

SELECT person_id FROM person WHERE is_team_leader AND team_id=?1;


c2: Partial Indexes
c3: 3. Queries Using Partial Indexes
c5: partialindex.html#queries_using_partial_indexes
c4:
Let X be the expression in the WHERE clause of a partial
index, and let W be the WHERE clause of a query that uses the
table that is indexed.  Then, the query is permitted to use 
the partial index if W&#x21d2;X, where the &#x21d2; operator
(usually pronounced "implies") is the logic operator 
equivalent to "X or not W".
Hence, determining whether or not a partial index
is usable in a particular query reduces to proving a theorem in
first-order logic.

SQLite does not have a sophisticated theorem
prover with which to determine W&#x21d2;X.  Instead, SQLite uses 
two simple rules to find the common cases where W&#x21d2;X is true, and
it assumes all the other cases are false.  The rules used by SQLite
are these:


If W is AND-connected terms and X is
OR-connected terms and if any term of W
appears as a term of X, then the partial index is usable.

For example, let the index be
CREATE INDEX ex1 ON tab1(a,b) WHERE a=5 OR b=6;

And let the query be:
SELECT * FROM tab1 WHERE b=6 AND a=7; -- uses partial index

Then the index is usable by the query because the "b=6" term appears
in both the index definition and in the query.  Remember: terms in the
index should be OR-connected and terms in the query should be AND-connected.

The terms in W and X must match exactly.  SQLite does not
do algebra to try to get them to look the same.
The term "b=6" does not match "b=3+3" or "b-6=0" or "b BETWEEN 6 AND 6".
"b=6" will match to "6=b" as long as "b=6" is on the index and "6=b" is
in the query.  If a term of the form "6=b" appears in the index, it will
never match anything.

If a term in X is of the form "z IS NOT NULL" and if a term in
       W is a comparison operator on "z" other than "IS", then those
       terms match.

Example:  Let the index be
CREATE INDEX ex2 ON tab2(b,c) WHERE c IS NOT NULL;

Then any query that uses operators =, &lt;, &gt;, &lt;=, &gt;=, &lt;&gt;,
IN, LIKE, or GLOB on column "c" 
would be usable with the partial index because those
comparison operators are only true if "c" is not NULL.  So the following
query could use the partial index:
SELECT * FROM tab2 WHERE b=456 AND c&lt;&gt;0;  -- uses partial index

But the next query can not use the partial index:
SELECT * FROM tab2 WHERE b=456;  -- cannot use partial index

The latter query can not use the partial index because there might be
rows in the table with b=456 and where c is NULL.  But those rows would
not be in the partial index.


These two rules describe how the query planner for SQLite works as of
this writing (2013-08-01).  And the rules above will always be honored.
However, future versions of SQLite might incorporate a better theorem prover
that can find other cases where W&#x21d2;X is true and thus may
find more instances where partial indexes are useful.


c2: Partial Indexes
c3: 4. Supported Versions
c5: partialindex.html#supported_versions
c4:
Partial indexes have been supported in SQLite since version 3.8.0
(2013-08-26).


Database files that contain partial indices are not readable or writable
by versions of SQLite prior to 3.8.0.  However, a database file created
by SQLite 3.8.0 is still readable and writable by prior versions as long
as its schema contains no partial indexes.  A database that is unreadable
by legacy versions of SQLite can be made readable simply by running
DROP INDEX on the partial indexes.
This page last modified on  2022-04-18 02:55:50 UTC


c2: Many Small Queries Are Efficient In SQLite
c3: 1. Executive Summary
c5: np1queryprob.html#executive_summary
c4:
200 SQL statements per webpage is excessive for client/server database
engines like MySQL, PostgreSQL, or SQL Server.


But with SQLite, 200 or more SQL statement per webpage is not a problem.


SQLite can also do large and complex queries efficiently, just like
client/server databases.  But SQLite can do many smaller queries
efficiently too.  Application developers can use whichever technique
works best for the task at hand.


c2: Many Small Queries Are Efficient In SQLite
c3: 2. The Perceived Problem
c5: np1queryprob.html#the_perceived_problem
c4:
The Appropriate Uses For SQLite page says that
dynamic pages on the SQLite website typically do about 200 SQL
statements each.
This has provoked criticism from readers.  Examples:



"200 SQL statements is a ridiculously high number for a single page"

"For most sites, 200 queries is way, way, way too much."

"&#91;This is&#93; bad design"



Such criticism would be well-founded for a traditional client/server
database engine, such as MySQL, PostgreSQL, or SQL Server.  In
a client/server database, each SQL statement requires a message
round-trip from the application to the database server and back to
the application.  Doing over 200 round-trip messages, sequentially,
can be a serious performance drag.  This is sometimes called the
"N+1 Query Problem" or the "N+1 Select Problem" and it is an anti-pattern.


c2: Many Small Queries Are Efficient In SQLite
c3: 3. N+1 Queries Are Not A Problem With SQLite
c5: np1queryprob.html#n_1_queries_are_not_a_problem_with_sqlite
c4:
SQLite is not client/server, however.  The SQLite database runs
in the same process address space as the application.  Queries do not
involve message round-trips, only a function call.  The latency
of a single SQL query is far less in SQLite.  Hence, using a large number
of queries with SQLite is not the problem.


c2: Many Small Queries Are Efficient In SQLite
c3: 4. The Need For Over 200 SQL Statements Per Webpage
c5: np1queryprob.html#the_need_for_over_200_sql_statements_per_webpage
c4:
The dynamic webpages on the SQLite website are mostly generated by
the Fossil version control system.
A typical dynamic
page would be a timeline such as https://www.sqlite.org/src/timeline.
A log of all SQL used by the timeline is shown below.


The first group of queries in the log are extracting display options
from the "config" and "global_config" tables of the Fossil database.
Then there is a single complex query that extracts a list of all elements
to be displayed on the timeline.
This "timeline" query demonstrates that SQLite can easily process complex
relational database queries involving multiple tables, subqueries, and
complex WHERE clause constraints, and it can make effective use of indexes
to solve the queries with minimal disk I/O.


Following the single big "timeline" query, 
there are additional queries for each timeline element.
Fossil is using the "N+1 Query" pattern rather than trying
to grab all the information in as few queries as possible.
But that is ok because there is no unnecessary IPC overhead.
At the bottom of
each timeline page, Fossil shows approximately how long it took to generate
the page.  For a 50-entry timeline, the latency is usually less than
25 milliseconds.  Profiling shows that few of those milliseconds
were spent inside the database engine.


Using the N+1 Query pattern in Fossil does not harm the application.  
But the N+1 Query pattern does have benefits.  For one, the
section of the code that creates the timeline query can be
completely separate from the section that prepares each timeline
entry for display.
This provides a separation of responsibility that helps keep the code
simple and easy to maintain.  Secondly, the information
needed for display, and the queries needed to extract that information,
vary according to what type of objects to be shown.  Check-ins need one
set of queries.  Tickets need another set of queries.  Wiki pages need a
different query.  And so forth.  By implementing these queries on-demand
and in the part of the code dealing with the various entities, there is
further separation of responsibility and simplification of the overall 
code base.


So, SQLite is able to do one or two large and complex queries, or it can
do many smaller and simpler queries.  Both are efficient.  An application
can use either or both techniques, depending on what works best for the
situation at hand.


The following is a log of all SQL used to generate one particular
timeline (captured on 2016-09-16):

-- sqlite3_open: /home/drh/sqlite/sqlite/.fslckout
PRAGMA foreign_keys=OFF;
SELECT sql FROM localdb.sqlite_schema WHERE name=='vfile';
-- sqlite3_open: /home/drh/.fossil
PRAGMA foreign_keys=OFF;
SELECT value FROM vvar WHERE name='repository';
ATTACH DATABASE '/home/drh/www/repos/sqlite.fossil' AS 'repository' KEY '';
SELECT value FROM config WHERE name='allow-symlinks';
SELECT value FROM global_config WHERE name='allow-symlinks';
SELECT value FROM config WHERE name='aux-schema';
SELECT 1 FROM config WHERE name='baseurl:http://';
SELECT value FROM config WHERE name='ip-prefix-terms';
SELECT value FROM global_config WHERE name='ip-prefix-terms';
SELECT value FROM config WHERE name='localauth';
SELECT value FROM vvar WHERE name='default-user';
SELECT uid FROM user WHERE cap LIKE '%s%';
SELECT login FROM user WHERE uid=1;
SELECT cap FROM user WHERE login = 'nobody';
SELECT cap FROM user WHERE login = 'anonymous';
SELECT value FROM config WHERE name='public-pages';
SELECT value FROM global_config WHERE name='public-pages';
SELECT value FROM config WHERE name='header';
SELECT value FROM config WHERE name='project-name';
SELECT value FROM config WHERE name='th1-setup';
SELECT value FROM global_config WHERE name='th1-setup';
SELECT value FROM config WHERE name='redirect-to-https';
SELECT value FROM global_config WHERE name='redirect-to-https';
SELECT value FROM config WHERE name='index-page';
SELECT mtime FROM config WHERE name='css';
SELECT mtime FROM config WHERE name='logo-image';
SELECT mtime FROM config WHERE name='background-image';
CREATE TEMP TABLE IF NOT EXISTS timeline(
  rid INTEGER PRIMARY KEY,
  uuid TEXT,
  timestamp TEXT,
  comment TEXT,
  user TEXT,
  isleaf BOOLEAN,
  bgcolor TEXT,
  etype TEXT,
  taglist TEXT,
  tagid INTEGER,
  short TEXT,
  sortby REAL
)
;
INSERT OR IGNORE INTO timeline SELECT
  blob.rid AS blobRid,
  uuid AS uuid,
  datetime(event.mtime,toLocal()) AS timestamp,
  coalesce(ecomment, comment) AS comment,
  coalesce(euser, user) AS user,
  blob.rid IN leaf AS leaf,
  bgcolor AS bgColor,
  event.type AS eventType,
  (SELECT group_concat(substr(tagname,5), ', ') FROM tag, tagxref
    WHERE tagname GLOB 'sym-*' AND tag.tagid=tagxref.tagid
      AND tagxref.rid=blob.rid AND tagxref.tagtype&gt;0) AS tags,
  tagid AS tagid,
  brief AS brief,
  event.mtime AS mtime
 FROM event CROSS JOIN blob
WHERE blob.rid=event.objid
 AND NOT EXISTS(SELECT 1 FROM tagxref WHERE tagid=5 AND tagtype&gt;0 AND rid=blob.rid)
 ORDER BY event.mtime DESC LIMIT 50;
-- SELECT value FROM config WHERE name='timeline-utc';
SELECT count(*) FROM timeline WHERE etype!='div';
SELECT min(timestamp) FROM timeline;
SELECT julianday('2016-09-15 14:54:51',fromLocal());
SELECT EXISTS (SELECT 1 FROM event CROSS JOIN blob WHERE blob.rid=event.objid AND mtime&lt;=2457647.121412037);
SELECT max(timestamp) FROM timeline;
SELECT julianday('2016-09-24 17:42:43',fromLocal());
SELECT EXISTS (SELECT 1 FROM event CROSS JOIN blob WHERE blob.rid=event.objid AND mtime&gt;=2457656.238009259);
SELECT value FROM config WHERE name='search-ci';
SELECT value FROM vvar WHERE name='checkout';
SELECT value FROM config WHERE name='timeline-max-comment';
SELECT value FROM global_config WHERE name='timeline-max-comment';
SELECT value FROM config WHERE name='timeline-date-format';
SELECT value FROM config WHERE name='timeline-truncate-at-blank';
SELECT value FROM global_config WHERE name='timeline-truncate-at-blank';
SELECT * FROM timeline ORDER BY sortby DESC;
SELECT value FROM config WHERE name='hash-digits';
SELECT value FROM global_config WHERE name='hash-digits';
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=68028;
SELECT pid FROM plink WHERE cid=68028 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM tagxref WHERE rid=68028 AND tagid=9 AND tagtype&gt;0;
SELECT value FROM config WHERE name='timeline-block-markup';
SELECT value FROM config WHERE name='timeline-plaintext';
SELECT value FROM config WHERE name='wiki-use-html';
SELECT value FROM global_config WHERE name='wiki-use-html';
SELECT 1 FROM private WHERE rid=68028;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=68026;
SELECT pid FROM plink WHERE cid=68026 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=68026;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=68024;
SELECT pid FROM plink WHERE cid=68024 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=68024;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=68018;
SELECT pid FROM plink WHERE cid=68018 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=68018;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=68012;
SELECT pid FROM plink WHERE cid=68012 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=68012;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=68011;
SELECT value FROM config WHERE name='details';
SELECT pid FROM plink WHERE cid=68011 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM tagxref WHERE rid=68011 AND tagid=9 AND tagtype&gt;0;
SELECT 1 FROM private WHERE rid=68011;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=68008;
SELECT pid FROM plink WHERE cid=68008 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=68008;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=68006;
SELECT pid FROM plink WHERE cid=68006 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=68006;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=68000;
SELECT pid FROM plink WHERE cid=68000 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=68000;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67997;
SELECT pid FROM plink WHERE cid=67997 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67997;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67992;
SELECT pid FROM plink WHERE cid=67992 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67992;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67990;
SELECT pid FROM plink WHERE cid=67990 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67990;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67989;
SELECT pid FROM plink WHERE cid=67989 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67989;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67984;
SELECT pid FROM plink WHERE cid=67984 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67984;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67983;
SELECT pid FROM plink WHERE cid=67983 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67983;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67979;
SELECT pid FROM plink WHERE cid=67979 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67979;
SELECT value FROM config WHERE name='ticket-closed-expr';
SELECT status='Closed' OR status='Fixed' FROM ticket  WHERE tkt_uuid&gt;='1ec41379c9c1e400' AND tkt_uuid&lt;'1ec41379c9c1e401';
SELECT 1 FROM private WHERE rid=67980;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67977;
SELECT pid FROM plink WHERE cid=67977 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT status='Closed' OR status='Fixed' FROM ticket  WHERE tkt_uuid&gt;='1ec41379c9c1e400' AND tkt_uuid&lt;'1ec41379c9c1e401';
SELECT 1 FROM private WHERE rid=67977;
SELECT status='Closed' OR status='Fixed' FROM ticket  WHERE tkt_uuid&gt;='1ec41379c9c1e400' AND tkt_uuid&lt;'1ec41379c9c1e401';
SELECT 1 FROM private WHERE rid=67974;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67971;
SELECT pid FROM plink WHERE cid=67971 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67971;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67972;
SELECT pid FROM plink WHERE cid=67972 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67972;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67969;
SELECT pid FROM plink WHERE cid=67969 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67969;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67966;
SELECT pid FROM plink WHERE cid=67966 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67966;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67962;
SELECT pid FROM plink WHERE cid=67962 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67962;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67960;
SELECT pid FROM plink WHERE cid=67960 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67960;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67957;
SELECT pid FROM plink WHERE cid=67957 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67957;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67955;
SELECT pid FROM plink WHERE cid=67955 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67955;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67953;
SELECT pid FROM plink WHERE cid=67953 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT status='Closed' OR status='Fixed' FROM ticket  WHERE tkt_uuid&gt;='5990a1bdb4a073' AND tkt_uuid&lt;'5990a1bdb4a074';
SELECT 1 FROM blob WHERE uuid&gt;='5990a1bdb4a073' AND uuid&lt;'5990a1bdb4a074';
SELECT 1 FROM private WHERE rid=67953;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67941;
SELECT pid FROM plink WHERE cid=67941 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67941;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67940;
SELECT pid FROM plink WHERE cid=67940 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67940;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67938;
SELECT pid FROM plink WHERE cid=67938 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67938;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67935;
SELECT pid FROM plink WHERE cid=67935 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67935;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67934;
SELECT pid FROM plink WHERE cid=67934 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67934;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67932;
SELECT pid FROM plink WHERE cid=67932 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67932;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67930;
SELECT pid FROM plink WHERE cid=67930 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67930;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67928;
SELECT pid FROM plink WHERE cid=67928 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM tagxref WHERE rid=67928 AND tagid=9 AND tagtype&gt;0;
SELECT 1 FROM private WHERE rid=67928;
SELECT status='Closed' OR status='Fixed' FROM ticket  WHERE tkt_uuid&gt;='0eab1ac7591f511d' AND tkt_uuid&lt;'0eab1ac7591f511e';
SELECT 1 FROM private WHERE rid=67919;
SELECT status='Closed' OR status='Fixed' FROM ticket  WHERE tkt_uuid&gt;='01874d252ac44861' AND tkt_uuid&lt;'01874d252ac44862';
SELECT 1 FROM blob WHERE uuid&gt;='01874d252ac44861' AND uuid&lt;'01874d252ac44862';
SELECT 1 FROM private WHERE rid=67918;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67916;
SELECT pid FROM plink WHERE cid=67916 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT status='Closed' OR status='Fixed' FROM ticket  WHERE tkt_uuid&gt;='0eab1ac759' AND tkt_uuid&lt;'0eab1ac75:';
SELECT 1 FROM private WHERE rid=67916;
SELECT status='Closed' OR status='Fixed' FROM ticket  WHERE tkt_uuid&gt;='a49bc0a8244feb08' AND tkt_uuid&lt;'a49bc0a8244feb09';
SELECT 1 FROM blob WHERE uuid&gt;='a49bc0a8244feb08' AND uuid&lt;'a49bc0a8244feb09';
SELECT 1 FROM private WHERE rid=67914;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67913;
SELECT pid FROM plink WHERE cid=67913 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT status='Closed' OR status='Fixed' FROM ticket  WHERE tkt_uuid&gt;='0eab1ac7591f' AND tkt_uuid&lt;'0eab1ac7591g';
SELECT 1 FROM private WHERE rid=67913;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67911;
SELECT pid FROM plink WHERE cid=67911 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67911;
SELECT status='Closed' OR status='Fixed' FROM ticket  WHERE tkt_uuid&gt;='0eab1ac7591f511d' AND tkt_uuid&lt;'0eab1ac7591f511e';
SELECT 1 FROM private WHERE rid=67909;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67907;
SELECT pid FROM plink WHERE cid=67907 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67907;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67899;
SELECT pid FROM plink WHERE cid=67899 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67899;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67897;
SELECT pid FROM plink WHERE cid=67897 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67897;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67895;
SELECT pid FROM plink WHERE cid=67895 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67895;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67893;
SELECT pid FROM plink WHERE cid=67893 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67893;
SELECT value FROM tagxref WHERE tagid=8 AND tagtype&gt;0 AND rid=67891;
SELECT pid FROM plink WHERE cid=67891 AND pid NOT IN phantom ORDER BY isprim DESC;
SELECT 1 FROM private WHERE rid=67891;
SELECT count(*) FROM plink
 WHERE pid=67928 AND isprim
   AND coalesce((SELECT value FROM tagxref
                  WHERE tagid=8 AND rid=plink.pid), 'trunk')
      =coalesce((SELECT value FROM tagxref
                  WHERE tagid=8 AND rid=plink.cid), 'trunk')
;
SELECT count(*) FROM plink
 WHERE pid=68011 AND isprim
   AND coalesce((SELECT value FROM tagxref
                  WHERE tagid=8 AND rid=plink.pid), 'trunk')
      =coalesce((SELECT value FROM tagxref
                  WHERE tagid=8 AND rid=plink.cid), 'trunk')
;
SELECT count(*) FROM plink
 WHERE pid=68028 AND isprim
   AND coalesce((SELECT value FROM tagxref
                  WHERE tagid=8 AND rid=plink.pid), 'trunk')
      =coalesce((SELECT value FROM tagxref
                  WHERE tagid=8 AND rid=plink.cid), 'trunk')
;
SELECT value FROM config WHERE name='show-version-diffs';
SELECT value FROM config WHERE name='adunit-omit-if-admin';
SELECT value FROM global_config WHERE name='adunit-omit-if-admin';
SELECT value FROM config WHERE name='adunit-omit-if-user';
SELECT value FROM global_config WHERE name='adunit-omit-if-user';
SELECT value FROM config WHERE name='adunit';
SELECT value FROM global_config WHERE name='adunit';
SELECT value FROM config WHERE name='auto-hyperlink-delay';
SELECT value FROM global_config WHERE name='auto-hyperlink-delay';
SELECT value FROM config WHERE name='footer';
PRAGMA database_list;
PRAGMA database_list;
PRAGMA localdb.freelist_count;
PRAGMA localdb.page_count;

This page last modified on  2022-01-08 05:02:57 UTC


c2: The sqlite3_analyzer.exe Utility Program
c3: 1. The sqlite3_analyzer.exe Utility Program
c5: sqlanalyze.html#the_sqlite3_analyzer_exe_utility_program
c4:
The sqlite3_analyzer.exe binary is a command-line utility program
that measures and displays how much and how efficiently space is used by 
individual tables and indexes with an SQLite database file.
Example usage:


sqlite3_analyzer database.sqlite



The output is a human-readable ASCII text report that provides information
on the space utilization of the database file.  The report is intended to
be self-explanatory, though there is some 
additional explanation of the
various parameters reported toward the end of the report.


The output is also valid SQL.  Most of the report text is contained within
a header comment, with various SQL statements that create and initialize
a database at the
end of the report.  The constructed database contains
the raw data from which the report was extracted.  Hence the original
report can be read into an instance of the command-line shell and then
the raw data can be queried to dig deeper into the space utilization of
a particular database file.


c2: The sqlite3_analyzer.exe Utility Program
c3: 1.1. Implementation
c5: sqlanalyze.html#implementation
c4:
The sqlite3_analyzer.exe program is a
TCL program that uses the dbstat virtual table
to gather information about the database file and then format that
information neatly.


c2: The sqlite3_analyzer.exe Utility Program
c3: 1.2. Example Output
c5: sqlanalyze.html#example_output
c4:
The following is sqlite3_analyzer output for an example
places.sqlite database used by Firefox.


/** Disk-Space Utilization Report For &#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;/places.sqlite

Page size in bytes................................ 32768     
Pages in the whole file (measured)................ 221       
Pages in the whole file (calculated).............. 221       
Pages that store data............................. 221        100.0% 
Pages on the freelist (per header)................ 0            0.0% 
Pages on the freelist (calculated)................ 0            0.0% 
Pages of auto-vacuum overhead..................... 0            0.0% 
Number of tables in the database.................. 14        
Number of indices................................. 23        
Number of defined indices......................... 17        
Number of implied indices......................... 6         
Size of the file in bytes......................... 7241728   
Bytes of user payload stored...................... 2503069     34.6% 

*** Page counts for all tables with their indices *****************************

MOZ_PLACES........................................ 142         64.3% 
MOZ_HISTORYVISITS................................. 41          18.6% 
MOZ_FAVICONS...................................... 15           6.8% 
MOZ_BOOKMARKS..................................... 5            2.3% 
MOZ_KEYWORDS...................................... 3            1.4% 
MOZ_ANNO_ATTRIBUTES............................... 2            0.90% 
MOZ_ANNOS......................................... 2            0.90% 
MOZ_BOOKMARKS_ROOTS............................... 2            0.90% 
MOZ_HOSTS......................................... 2            0.90% 
MOZ_INPUTHISTORY.................................. 2            0.90% 
MOZ_ITEMS_ANNOS................................... 2            0.90% 
SQLITE_SCHEMA..................................... 1            0.45% 
SQLITE_SEQUENCE................................... 1            0.45% 
SQLITE_STAT1...................................... 1            0.45% 

*** Page counts for all tables and indices separately *************************

MOZ_PLACES........................................ 63          28.5% 
MOZ_PLACES_URL_UNIQUEINDEX........................ 37          16.7% 
MOZ_HISTORYVISITS................................. 13           5.9% 
MOZ_FAVICONS...................................... 12           5.4% 
MOZ_HISTORYVISITS_PLACEDATEINDEX.................. 12           5.4% 
MOZ_PLACES_HOSTINDEX.............................. 11           5.0% 
MOZ_HISTORYVISITS_DATEINDEX....................... 10           4.5% 
MOZ_PLACES_GUID_UNIQUEINDEX....................... 9            4.1% 
MOZ_PLACES_LASTVISITDATEINDEX..................... 7            3.2% 
MOZ_HISTORYVISITS_FROMINDEX....................... 6            2.7% 
MOZ_PLACES_FAVICONINDEX........................... 5            2.3% 
MOZ_PLACES_FRECENCYINDEX.......................... 5            2.3% 
MOZ_PLACES_VISITCOUNT............................. 5            2.3% 
SQLITE_AUTOINDEX_MOZ_FAVICONS_1................... 3            1.4% 
MOZ_ANNO_ATTRIBUTES............................... 1            0.45% 
MOZ_ANNOS......................................... 1            0.45% 
MOZ_ANNOS_PLACEATTRIBUTEINDEX..................... 1            0.45% 
MOZ_BOOKMARKS..................................... 1            0.45% 
MOZ_BOOKMARKS_GUID_UNIQUEINDEX.................... 1            0.45% 
MOZ_BOOKMARKS_ITEMINDEX........................... 1            0.45% 
MOZ_BOOKMARKS_ITEMLASTMODIFIEDINDEX............... 1            0.45% 
MOZ_BOOKMARKS_PARENTINDEX......................... 1            0.45% 
MOZ_BOOKMARKS_ROOTS............................... 1            0.45% 
MOZ_HOSTS......................................... 1            0.45% 
MOZ_INPUTHISTORY.................................. 1            0.45% 
MOZ_ITEMS_ANNOS................................... 1            0.45% 
MOZ_ITEMS_ANNOS_ITEMATTRIBUTEINDEX................ 1            0.45% 
MOZ_KEYWORDS...................................... 1            0.45% 
MOZ_KEYWORDS_PLACEPOSTDATA_UNIQUEINDEX............ 1            0.45% 
SQLITE_AUTOINDEX_MOZ_ANNO_ATTRIBUTES_1............ 1            0.45% 
SQLITE_AUTOINDEX_MOZ_BOOKMARKS_ROOTS_1............ 1            0.45% 
SQLITE_AUTOINDEX_MOZ_HOSTS_1...................... 1            0.45% 
SQLITE_AUTOINDEX_MOZ_INPUTHISTORY_1............... 1            0.45% 
SQLITE_AUTOINDEX_MOZ_KEYWORDS_1................... 1            0.45% 
SQLITE_SCHEMA..................................... 1            0.45% 
SQLITE_SEQUENCE................................... 1            0.45% 
SQLITE_STAT1...................................... 1            0.45% 

*** All tables and indices ****************************************************

Percentage of total database...................... 100.0%    
Number of entries................................. 154969    
Bytes of storage consumed......................... 7241728   
Bytes of payload.................................. 4969404     68.6% 
Average payload per entry......................... 32.07     
Average unused bytes per entry.................... 11.15     
Average fanout.................................... 14.00     
Maximum payload per entry......................... 7640      
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 14        
Primary pages used................................ 207       
Overflow pages used............................... 0         
Total pages used.................................. 221       
Unused bytes on index pages....................... 448010      97.7% 
Unused bytes on primary pages..................... 1280642     18.9% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 1728652     23.9% 

*** All tables ****************************************************************

Percentage of total database......................  44.8%    
Number of entries................................. 28530     
Bytes of storage consumed......................... 3244032   
Bytes of payload.................................. 2508257     77.3% 
Average payload per entry......................... 87.92     
Average unused bytes per entry.................... 20.13     
Average fanout.................................... 28.00     
Maximum payload per entry......................... 7640      
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 3         
Primary pages used................................ 96        
Overflow pages used............................... 0         
Total pages used.................................. 99        
Unused bytes on index pages....................... 97551       99.23% 
Unused bytes on primary pages..................... 476741      15.2% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 574292      17.7% 

*** All indices ***************************************************************

Percentage of total database......................  55.2%    
Number of entries................................. 126439    
Bytes of storage consumed......................... 3997696   
Bytes of payload.................................. 2461147     61.6% 
Average payload per entry......................... 19.47     
Average unused bytes per entry.................... 9.13      
Average fanout.................................... 11.00     
Maximum payload per entry......................... 7259      
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 11        
Primary pages used................................ 111       
Overflow pages used............................... 0         
Total pages used.................................. 122       
Unused bytes on index pages....................... 350459      97.2% 
Unused bytes on primary pages..................... 803901      22.1% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 1154360     28.9% 

*** Table MOZ_ANNO_ATTRIBUTES and all its indices *****************************

Percentage of total database......................   0.90%   
Number of entries................................. 24        
Bytes of storage consumed......................... 65536     
Bytes of payload.................................. 721          1.1% 
Average payload per entry......................... 30.04     
Average unused bytes per entry.................... 2696.46   
Maximum payload per entry......................... 43        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 2         
Overflow pages used............................... 0         
Total pages used.................................. 2         
Unused bytes on primary pages..................... 64715       98.7% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 64715       98.7% 

*** Table MOZ_ANNO_ATTRIBUTES w/o any indices *********************************

Percentage of total database......................   0.45%   
Number of entries................................. 12        
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 355          1.1% 
B-tree depth...................................... 1         
Average payload per entry......................... 29.58     
Average unused bytes per entry.................... 2696.42   
Maximum payload per entry......................... 42        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 32357       98.7% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 32357       98.7% 

*** Index SQLITE_AUTOINDEX_MOZ_ANNO_ATTRIBUTES_1 of table MOZ_ANNO_ATTRIBUTES *

Percentage of total database......................   0.45%   
Number of entries................................. 12        
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 366          1.1% 
B-tree depth...................................... 1         
Average payload per entry......................... 30.50     
Average unused bytes per entry.................... 2696.50   
Maximum payload per entry......................... 43        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 32358       98.7% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 32358       98.7% 

*** Table MOZ_ANNOS and all its indices ***************************************

Percentage of total database......................   0.90%   
Number of entries................................. 390       
Bytes of storage consumed......................... 65536     
Bytes of payload.................................. 13986       21.3% 
Average payload per entry......................... 35.86     
Average unused bytes per entry.................... 128.22    
Maximum payload per entry......................... 127       
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 2         
Overflow pages used............................... 0         
Total pages used.................................. 2         
Unused bytes on primary pages..................... 50006       76.3% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 50006       76.3% 

*** Table MOZ_ANNOS w/o any indices *******************************************

Percentage of total database......................   0.45%   
Number of entries................................. 195       
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 12115       37.0% 
B-tree depth...................................... 1         
Average payload per entry......................... 62.13     
Average unused bytes per entry.................... 101.04    
Maximum payload per entry......................... 127       
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 19702       60.1% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 19702       60.1% 

*** Index MOZ_ANNOS_PLACEATTRIBUTEINDEX of table MOZ_ANNOS ********************

Percentage of total database......................   0.45%   
Number of entries................................. 195       
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 1871         5.7% 
B-tree depth...................................... 1         
Average payload per entry......................... 9.59      
Average unused bytes per entry.................... 155.41    
Maximum payload per entry......................... 10        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 30304       92.5% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 30304       92.5% 

*** Table MOZ_BOOKMARKS and all its indices ***********************************

Percentage of total database......................   2.3%    
Number of entries................................. 1565      
Bytes of storage consumed......................... 163840    
Bytes of payload.................................. 37104       22.6% 
Average payload per entry......................... 23.71     
Average unused bytes per entry.................... 77.62     
Maximum payload per entry......................... 518       
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 5         
Overflow pages used............................... 0         
Total pages used.................................. 5         
Unused bytes on primary pages..................... 121475      74.1% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 121475      74.1% 

*** Table MOZ_BOOKMARKS w/o any indices ***************************************

Percentage of total database......................   0.45%   
Number of entries................................. 313       
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 21937       66.9% 
B-tree depth...................................... 1         
Average payload per entry......................... 70.09     
Average unused bytes per entry.................... 29.90     
Maximum payload per entry......................... 518       
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 9358        28.6% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 9358        28.6% 

*** Indices of table MOZ_BOOKMARKS ********************************************

Percentage of total database......................   1.8%    
Number of entries................................. 1252      
Bytes of storage consumed......................... 131072    
Bytes of payload.................................. 15167       11.6% 
Average payload per entry......................... 12.11     
Average unused bytes per entry.................... 89.55     
Maximum payload per entry......................... 17        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 4         
Overflow pages used............................... 0         
Total pages used.................................. 4         
Unused bytes on primary pages..................... 112117      85.5% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 112117      85.5% 

*** Index MOZ_BOOKMARKS_GUID_UNIQUEINDEX of table MOZ_BOOKMARKS ***************

Percentage of total database......................   0.45%   
Number of entries................................. 313       
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 5207        15.9% 
B-tree depth...................................... 1         
Average payload per entry......................... 16.64     
Average unused bytes per entry.................... 85.03     
Maximum payload per entry......................... 17        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 26614       81.2% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 26614       81.2% 

*** Index MOZ_BOOKMARKS_ITEMINDEX of table MOZ_BOOKMARKS **********************

Percentage of total database......................   0.45%   
Number of entries................................. 313       
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 2547         7.8% 
B-tree depth...................................... 1         
Average payload per entry......................... 8.14      
Average unused bytes per entry.................... 93.53     
Maximum payload per entry......................... 9         
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 29274       89.3% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 29274       89.3% 

*** Index MOZ_BOOKMARKS_ITEMLASTMODIFIEDINDEX of table MOZ_BOOKMARKS **********

Percentage of total database......................   0.45%   
Number of entries................................. 313       
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 5020        15.3% 
B-tree depth...................................... 1         
Average payload per entry......................... 16.04     
Average unused bytes per entry.................... 85.63     
Maximum payload per entry......................... 17        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 26801       81.8% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 26801       81.8% 

*** Index MOZ_BOOKMARKS_PARENTINDEX of table MOZ_BOOKMARKS ********************

Percentage of total database......................   0.45%   
Number of entries................................. 313       
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 2393         7.3% 
B-tree depth...................................... 1         
Average payload per entry......................... 7.65      
Average unused bytes per entry.................... 94.02     
Maximum payload per entry......................... 9         
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 29428       89.8% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 29428       89.8% 

*** Table MOZ_BOOKMARKS_ROOTS and all its indices *****************************

Percentage of total database......................   0.90%   
Number of entries................................. 10        
Bytes of storage consumed......................... 65536     
Bytes of payload.................................. 94           0.14% 
Average payload per entry......................... 9.40      
Average unused bytes per entry.................... 6539.10   
Maximum payload per entry......................... 11        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 2         
Overflow pages used............................... 0         
Total pages used.................................. 2         
Unused bytes on primary pages..................... 65391       99.78% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 65391       99.78% 

*** Table MOZ_BOOKMARKS_ROOTS w/o any indices *********************************

Percentage of total database......................   0.45%   
Number of entries................................. 5         
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 47           0.14% 
B-tree depth...................................... 1         
Average payload per entry......................... 9.40      
Average unused bytes per entry.................... 6538.60   
Maximum payload per entry......................... 11        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 32693       99.77% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 32693       99.77% 

*** Index SQLITE_AUTOINDEX_MOZ_BOOKMARKS_ROOTS_1 of table MOZ_BOOKMARKS_ROOTS *

Percentage of total database......................   0.45%   
Number of entries................................. 5         
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 47           0.14% 
B-tree depth...................................... 1         
Average payload per entry......................... 9.40      
Average unused bytes per entry.................... 6539.60   
Maximum payload per entry......................... 11        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 32698       99.79% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 32698       99.79% 

*** Table MOZ_FAVICONS and all its indices ************************************

Percentage of total database......................   6.8%    
Number of entries................................. 941       
Bytes of storage consumed......................... 491520    
Bytes of payload.................................. 332765      67.7% 
Average payload per entry......................... 353.63    
Average unused bytes per entry.................... 164.00    
Average fanout.................................... 7.00      
Maximum payload per entry......................... 7640      
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 2         
Primary pages used................................ 13        
Overflow pages used............................... 0         
Total pages used.................................. 15        
Unused bytes on index pages....................... 65340       99.70% 
Unused bytes on primary pages..................... 88980       20.9% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 154320      31.4% 

*** Table MOZ_FAVICONS w/o any indices ****************************************

Percentage of total database......................   5.4%    
Number of entries................................. 471       
Bytes of storage consumed......................... 393216    
Bytes of payload.................................. 297630      75.7% 
B-tree depth...................................... 2         
Average payload per entry......................... 631.91    
Average unused bytes per entry.................... 196.60    
Average fanout.................................... 11.00     
Non-sequential pages.............................. 6           54.5% 
Maximum payload per entry......................... 7640      
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1         
Primary pages used................................ 11        
Overflow pages used............................... 0         
Total pages used.................................. 12        
Unused bytes on index pages....................... 32676       99.72% 
Unused bytes on primary pages..................... 59923       16.6% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 92599       23.5% 

*** Index SQLITE_AUTOINDEX_MOZ_FAVICONS_1 of table MOZ_FAVICONS ***************

Percentage of total database......................   1.4%    
Number of entries................................. 470       
Bytes of storage consumed......................... 98304     
Bytes of payload.................................. 35135       35.7% 
B-tree depth...................................... 2         
Average payload per entry......................... 74.76     
Average unused bytes per entry.................... 131.32    
Average fanout.................................... 3.00      
Non-sequential pages.............................. 1           50.0% 
Maximum payload per entry......................... 7259      
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1         
Primary pages used................................ 2         
Overflow pages used............................... 0         
Total pages used.................................. 3         
Unused bytes on index pages....................... 32664       99.68% 
Unused bytes on primary pages..................... 29057       44.3% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 61721       62.8% 

*** Table MOZ_HISTORYVISITS and all its indices *******************************

Percentage of total database......................  18.6%    
Number of entries................................. 63470     
Bytes of storage consumed......................... 1343488   
Bytes of payload.................................. 882233      65.7% 
Average payload per entry......................... 13.90     
Average unused bytes per entry.................... 3.76      
Average fanout.................................... 10.00     
Maximum payload per entry......................... 21        
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 4         
Primary pages used................................ 37        
Overflow pages used............................... 0         
Total pages used.................................. 41        
Unused bytes on index pages....................... 130482      99.55% 
Unused bytes on primary pages..................... 108158       8.9% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 238640      17.8% 

*** Table MOZ_HISTORYVISITS w/o any indices ***********************************

Percentage of total database......................   5.9%    
Number of entries................................. 15873     
Bytes of storage consumed......................... 425984    
Bytes of payload.................................. 308447      72.4% 
B-tree depth...................................... 2         
Average payload per entry......................... 19.43     
Average unused bytes per entry.................... 2.40      
Average fanout.................................... 12.00     
Non-sequential pages.............................. 8           66.7% 
Maximum payload per entry......................... 21        
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1         
Primary pages used................................ 12        
Overflow pages used............................... 0         
Total pages used.................................. 13        
Unused bytes on index pages....................... 32668       99.69% 
Unused bytes on primary pages..................... 5435         1.4% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 38103        8.9% 

*** Indices of table MOZ_HISTORYVISITS ****************************************

Percentage of total database......................  12.7%    
Number of entries................................. 47597     
Bytes of storage consumed......................... 917504    
Bytes of payload.................................. 573786      62.5% 
Average payload per entry......................... 12.06     
Average unused bytes per entry.................... 4.21      
Average fanout.................................... 9.00      
Maximum payload per entry......................... 17        
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 3         
Primary pages used................................ 25        
Overflow pages used............................... 0         
Total pages used.................................. 28        
Unused bytes on index pages....................... 97814       99.50% 
Unused bytes on primary pages..................... 102723      12.5% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 200537      21.9% 

*** Index MOZ_HISTORYVISITS_DATEINDEX of table MOZ_HISTORYVISITS **************

Percentage of total database......................   4.5%    
Number of entries................................. 15865     
Bytes of storage consumed......................... 327680    
Bytes of payload.................................. 206221      62.9% 
B-tree depth...................................... 2         
Average payload per entry......................... 13.00     
Average unused bytes per entry.................... 4.65      
Average fanout.................................... 10.00     
Non-sequential pages.............................. 6           66.7% 
Maximum payload per entry......................... 13        
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1         
Primary pages used................................ 9         
Overflow pages used............................... 0         
Total pages used.................................. 10        
Unused bytes on index pages....................... 32596       99.48% 
Unused bytes on primary pages..................... 41128       13.9% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 73724       22.5% 

*** Index MOZ_HISTORYVISITS_FROMINDEX of table MOZ_HISTORYVISITS **************

Percentage of total database......................   2.7%    
Number of entries................................. 15869     
Bytes of storage consumed......................... 196608    
Bytes of payload.................................. 100292      51.0% 
B-tree depth...................................... 2         
Average payload per entry......................... 6.32      
Average unused bytes per entry.................... 3.06      
Average fanout.................................... 6.00      
Non-sequential pages.............................. 4           80.0% 
Maximum payload per entry......................... 7         
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1         
Primary pages used................................ 5         
Overflow pages used............................... 0         
Total pages used.................................. 6         
Unused bytes on index pages....................... 32702       99.80% 
Unused bytes on primary pages..................... 15927        9.7% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 48629       24.7% 

*** Index MOZ_HISTORYVISITS_PLACEDATEINDEX of table MOZ_HISTORYVISITS *********

Percentage of total database......................   5.4%    
Number of entries................................. 15863     
Bytes of storage consumed......................... 393216    
Bytes of payload.................................. 267273      68.0% 
B-tree depth...................................... 2         
Average payload per entry......................... 16.85     
Average unused bytes per entry.................... 4.93      
Average fanout.................................... 12.00     
Non-sequential pages.............................. 8           72.7% 
Maximum payload per entry......................... 17        
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1         
Primary pages used................................ 11        
Overflow pages used............................... 0         
Total pages used.................................. 12        
Unused bytes on index pages....................... 32516       99.23% 
Unused bytes on primary pages..................... 45668       12.7% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 78184       19.9% 

*** Table MOZ_HOSTS and all its indices ***************************************

Percentage of total database......................   0.90%   
Number of entries................................. 1256      
Bytes of storage consumed......................... 65536     
Bytes of payload.................................. 27640       42.2% 
Average payload per entry......................... 22.01     
Average unused bytes per entry.................... 26.18     
Maximum payload per entry......................... 49        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 2         
Overflow pages used............................... 0         
Total pages used.................................. 2         
Unused bytes on primary pages..................... 32888       50.2% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 32888       50.2% 

*** Table MOZ_HOSTS w/o any indices *******************************************

Percentage of total database......................   0.45%   
Number of entries................................. 628       
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 14640       44.7% 
B-tree depth...................................... 1         
Average payload per entry......................... 23.31     
Average unused bytes per entry.................... 23.90     
Maximum payload per entry......................... 49        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 15012       45.8% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 15012       45.8% 

*** Index SQLITE_AUTOINDEX_MOZ_HOSTS_1 of table MOZ_HOSTS *********************

Percentage of total database......................   0.45%   
Number of entries................................. 628       
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 13000       39.7% 
B-tree depth...................................... 1         
Average payload per entry......................... 20.70     
Average unused bytes per entry.................... 28.46     
Maximum payload per entry......................... 47        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 17876       54.6% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 17876       54.6% 

*** Table MOZ_INPUTHISTORY and all its indices ********************************

Percentage of total database......................   0.90%   
Number of entries................................. 16        
Bytes of storage consumed......................... 65536     
Bytes of payload.................................. 642          0.98% 
Average payload per entry......................... 40.12     
Average unused bytes per entry.................... 4050.88   
Maximum payload per entry......................... 71        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 2         
Overflow pages used............................... 0         
Total pages used.................................. 2         
Unused bytes on primary pages..................... 64814       98.9% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 64814       98.9% 

*** Table MOZ_INPUTHISTORY w/o any indices ************************************

Percentage of total database......................   0.45%   
Number of entries................................. 8         
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 341          1.0% 
B-tree depth...................................... 1         
Average payload per entry......................... 42.62     
Average unused bytes per entry.................... 4047.38   
Maximum payload per entry......................... 71        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 32379       98.8% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 32379       98.8% 

*** Index SQLITE_AUTOINDEX_MOZ_INPUTHISTORY_1 of table MOZ_INPUTHISTORY *******

Percentage of total database......................   0.45%   
Number of entries................................. 8         
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 301          0.92% 
B-tree depth...................................... 1         
Average payload per entry......................... 37.62     
Average unused bytes per entry.................... 4054.38   
Maximum payload per entry......................... 65        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 32435       99.0% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 32435       99.0% 

*** Table MOZ_ITEMS_ANNOS and all its indices *********************************

Percentage of total database......................   0.90%   
Number of entries................................. 158       
Bytes of storage consumed......................... 65536     
Bytes of payload.................................. 9211        14.1% 
Average payload per entry......................... 58.30     
Average unused bytes per entry.................... 352.56    
Maximum payload per entry......................... 384       
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 2         
Overflow pages used............................... 0         
Total pages used.................................. 2         
Unused bytes on primary pages..................... 55704       85.0% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 55704       85.0% 

*** Table MOZ_ITEMS_ANNOS w/o any indices *************************************

Percentage of total database......................   0.45%   
Number of entries................................. 79        
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 8649        26.4% 
B-tree depth...................................... 1         
Average payload per entry......................... 109.48    
Average unused bytes per entry.................... 300.54    
Maximum payload per entry......................... 384       
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 23743       72.5% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 23743       72.5% 

*** Index MOZ_ITEMS_ANNOS_ITEMATTRIBUTEINDEX of table MOZ_ITEMS_ANNOS *********

Percentage of total database......................   0.45%   
Number of entries................................. 79        
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 562          1.7% 
B-tree depth...................................... 1         
Average payload per entry......................... 7.11      
Average unused bytes per entry.................... 404.57    
Maximum payload per entry......................... 9         
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 31961       97.5% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 31961       97.5% 

*** Table MOZ_KEYWORDS and all its indices ************************************

Percentage of total database......................   1.4%    
Number of entries................................. 0         
Bytes of storage consumed......................... 98304     
Bytes of payload.................................. 0            0.0% 
Average payload per entry......................... 0.0       
Average unused bytes per entry.................... 0.0       
Maximum payload per entry......................... 0         
Entries that use overflow......................... 0         
Primary pages used................................ 3         
Overflow pages used............................... 0         
Total pages used.................................. 3         
Unused bytes on primary pages..................... 98280       99.976% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 98280       99.976% 

*** Table MOZ_KEYWORDS w/o any indices ****************************************

Percentage of total database......................   0.45%   
Number of entries................................. 0         
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 0            0.0% 
B-tree depth...................................... 1         
Average payload per entry......................... 0.0       
Average unused bytes per entry.................... 0.0       
Maximum payload per entry......................... 0         
Entries that use overflow......................... 0         
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 32760       99.976% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 32760       99.976% 

*** Indices of table MOZ_KEYWORDS *********************************************

Percentage of total database......................   0.90%   
Number of entries................................. 0         
Bytes of storage consumed......................... 65536     
Bytes of payload.................................. 0            0.0% 
Average payload per entry......................... 0.0       
Average unused bytes per entry.................... 0.0       
Maximum payload per entry......................... 0         
Entries that use overflow......................... 0         
Primary pages used................................ 2         
Overflow pages used............................... 0         
Total pages used.................................. 2         
Unused bytes on primary pages..................... 65520       99.976% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 65520       99.976% 

*** Index MOZ_KEYWORDS_PLACEPOSTDATA_UNIQUEINDEX of table MOZ_KEYWORDS ********

Percentage of total database......................   0.45%   
Number of entries................................. 0         
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 0            0.0% 
B-tree depth...................................... 1         
Average payload per entry......................... 0.0       
Average unused bytes per entry.................... 0.0       
Maximum payload per entry......................... 0         
Entries that use overflow......................... 0         
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 32760       99.976% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 32760       99.976% 

*** Index SQLITE_AUTOINDEX_MOZ_KEYWORDS_1 of table MOZ_KEYWORDS ***************

Percentage of total database......................   0.45%   
Number of entries................................. 0         
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 0            0.0% 
B-tree depth...................................... 1         
Average payload per entry......................... 0.0       
Average unused bytes per entry.................... 0.0       
Maximum payload per entry......................... 0         
Entries that use overflow......................... 0         
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 32760       99.976% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 32760       99.976% 

*** Table MOZ_PLACES and all its indices **************************************

Percentage of total database......................  64.3%    
Number of entries................................. 87087     
Bytes of storage consumed......................... 4653056   
Bytes of payload.................................. 3659043     78.6% 
Average payload per entry......................... 42.02     
Average unused bytes per entry.................... 7.93      
Average fanout.................................... 17.00     
Maximum payload per entry......................... 1867      
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 8         
Primary pages used................................ 134       
Overflow pages used............................... 0         
Total pages used.................................. 142       
Unused bytes on index pages....................... 252188      96.2% 
Unused bytes on primary pages..................... 438258      10.0% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 690446      14.8% 

*** Table MOZ_PLACES w/o any indices ******************************************

Percentage of total database......................  28.5%    
Number of entries................................. 10894     
Bytes of storage consumed......................... 2064384   
Bytes of payload.................................. 1838131     89.0% 
B-tree depth...................................... 2         
Average payload per entry......................... 168.73    
Average unused bytes per entry.................... 14.10     
Average fanout.................................... 62.00     
Non-sequential pages.............................. 30          48.4% 
Maximum payload per entry......................... 1867      
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1         
Primary pages used................................ 62        
Overflow pages used............................... 0         
Total pages used.................................. 63        
Unused bytes on index pages....................... 32207       98.3% 
Unused bytes on primary pages..................... 121406       6.0% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 153613       7.4% 

*** Indices of table MOZ_PLACES ***********************************************

Percentage of total database......................  35.7%    
Number of entries................................. 76193     
Bytes of storage consumed......................... 2588672   
Bytes of payload.................................. 1820912     70.3% 
Average payload per entry......................... 23.90     
Average unused bytes per entry.................... 7.05      
Average fanout.................................... 11.00     
Maximum payload per entry......................... 1823      
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 7         
Primary pages used................................ 72        
Overflow pages used............................... 0         
Total pages used.................................. 79        
Unused bytes on index pages....................... 219981      95.9% 
Unused bytes on primary pages..................... 316852      13.4% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 536833      20.7% 

*** Index MOZ_PLACES_FAVICONINDEX of table MOZ_PLACES *************************

Percentage of total database......................   2.3%    
Number of entries................................. 10891     
Bytes of storage consumed......................... 163840    
Bytes of payload.................................. 83178       50.8% 
B-tree depth...................................... 2         
Average payload per entry......................... 7.64      
Average unused bytes per entry.................... 4.40      
Average fanout.................................... 5.00      
Non-sequential pages.............................. 3           75.0% 
Maximum payload per entry......................... 8         
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1         
Primary pages used................................ 4         
Overflow pages used............................... 0         
Total pages used.................................. 5         
Unused bytes on index pages....................... 32711       99.83% 
Unused bytes on primary pages..................... 15213       11.6% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 47924       29.3% 

*** Index MOZ_PLACES_FRECENCYINDEX of table MOZ_PLACES ************************

Percentage of total database......................   2.3%    
Number of entries................................. 10891     
Bytes of storage consumed......................... 163840    
Bytes of payload.................................. 76772       46.9% 
B-tree depth...................................... 2         
Average payload per entry......................... 7.05      
Average unused bytes per entry.................... 4.99      
Average fanout.................................... 5.00      
Non-sequential pages.............................. 3           75.0% 
Maximum payload per entry......................... 9         
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1         
Primary pages used................................ 4         
Overflow pages used............................... 0         
Total pages used.................................. 5         
Unused bytes on index pages....................... 32714       99.84% 
Unused bytes on primary pages..................... 21616       16.5% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 54330       33.2% 

*** Index MOZ_PLACES_GUID_UNIQUEINDEX of table MOZ_PLACES *********************

Percentage of total database......................   4.1%    
Number of entries................................. 10887     
Bytes of storage consumed......................... 294912    
Bytes of payload.................................. 196000      66.5% 
B-tree depth...................................... 2         
Average payload per entry......................... 18.00     
Average unused bytes per entry.................... 6.07      
Average fanout.................................... 9.00      
Non-sequential pages.............................. 5           62.5% 
Maximum payload per entry......................... 18        
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1         
Primary pages used................................ 8         
Overflow pages used............................... 0         
Total pages used.................................. 9         
Unused bytes on index pages....................... 32581       99.43% 
Unused bytes on primary pages..................... 33545       12.8% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 66126       22.4% 

*** Index MOZ_PLACES_HOSTINDEX of table MOZ_PLACES ****************************

Percentage of total database......................   5.0%    
Number of entries................................. 10885     
Bytes of storage consumed......................... 360448    
Bytes of payload.................................. 237383      65.9% 
B-tree depth...................................... 2         
Average payload per entry......................... 21.81     
Average unused bytes per entry.................... 8.29      
Average fanout.................................... 11.00     
Non-sequential pages.............................. 7           70.0% 
Maximum payload per entry......................... 49        
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1         
Primary pages used................................ 10        
Overflow pages used............................... 0         
Total pages used.................................. 11        
Unused bytes on index pages....................... 32473       99.10% 
Unused bytes on primary pages..................... 57782       17.6% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 90255       25.0% 

*** Index MOZ_PLACES_LASTVISITDATEINDEX of table MOZ_PLACES *******************

Percentage of total database......................   3.2%    
Number of entries................................. 10889     
Bytes of storage consumed......................... 229376    
Bytes of payload.................................. 150784      65.7% 
B-tree depth...................................... 2         
Average payload per entry......................... 13.85     
Average unused bytes per entry.................... 4.21      
Average fanout.................................... 7.00      
Non-sequential pages.............................. 4           66.7% 
Maximum payload per entry......................... 14        
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1         
Primary pages used................................ 6         
Overflow pages used............................... 0         
Total pages used.................................. 7         
Unused bytes on index pages....................... 32651       99.64% 
Unused bytes on primary pages..................... 13179        6.7% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 45830       20.0% 

*** Index MOZ_PLACES_URL_UNIQUEINDEX of table MOZ_PLACES **********************

Percentage of total database......................  16.7%    
Number of entries................................. 10859     
Bytes of storage consumed......................... 1212416   
Bytes of payload.................................. 1010666     83.4% 
B-tree depth...................................... 2         
Average payload per entry......................... 93.07     
Average unused bytes per entry.................... 15.42     
Average fanout.................................... 37.00     
Non-sequential pages.............................. 16          44.4% 
Maximum payload per entry......................... 1823      
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1         
Primary pages used................................ 36        
Overflow pages used............................... 0         
Total pages used.................................. 37        
Unused bytes on index pages....................... 24134       73.7% 
Unused bytes on primary pages..................... 143261      12.1% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 167395      13.8% 

*** Index MOZ_PLACES_VISITCOUNT of table MOZ_PLACES ***************************

Percentage of total database......................   2.3%    
Number of entries................................. 10891     
Bytes of storage consumed......................... 163840    
Bytes of payload.................................. 66129       40.4% 
B-tree depth...................................... 2         
Average payload per entry......................... 6.07      
Average unused bytes per entry.................... 5.97      
Average fanout.................................... 5.00      
Non-sequential pages.............................. 3           75.0% 
Maximum payload per entry......................... 8         
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1         
Primary pages used................................ 4         
Overflow pages used............................... 0         
Total pages used.................................. 5         
Unused bytes on index pages....................... 32717       99.84% 
Unused bytes on primary pages..................... 32256       24.6% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 64973       39.7% 

*** Table SQLITE_SCHEMA *******************************************************

Percentage of total database......................   0.45%   
Number of entries................................. 36        
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 5188        15.8% 
B-tree depth...................................... 1         
Average payload per entry......................... 144.11    
Average unused bytes per entry.................... 758.58    
Maximum payload per entry......................... 379       
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 27309       83.3% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 27309       83.3% 

*** Table SQLITE_SEQUENCE *****************************************************

Percentage of total database......................   0.45%   
Number of entries................................. 1         
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 15           0.046% 
B-tree depth...................................... 1         
Average payload per entry......................... 15.00     
Average unused bytes per entry.................... 32741.00  
Maximum payload per entry......................... 15        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 32741       99.918% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 32741       99.918% 

*** Table SQLITE_STAT1 ********************************************************

Percentage of total database......................   0.45%   
Number of entries................................. 15        
Bytes of storage consumed......................... 32768     
Bytes of payload.................................. 762          2.3% 
B-tree depth...................................... 1         
Average payload per entry......................... 50.80     
Average unused bytes per entry.................... 2128.20   
Maximum payload per entry......................... 62        
Entries that use overflow......................... 0            0.0% 
Primary pages used................................ 1         
Overflow pages used............................... 0         
Total pages used.................................. 1         
Unused bytes on primary pages..................... 31923       97.4% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 31923       97.4% 

*** Definitions ***************************************************************

Page size in bytes

    The number of bytes in a single page of the database file.  
    Usually 1024.

Number of pages in the whole file

    The number of 32768-byte pages that go into forming the complete
    database

Pages that store data

    The number of pages that store data, either as primary B*Tree pages or
    as overflow pages.  The number at the right is the data pages divided by
    the total number of pages in the file.

Pages on the freelist

    The number of pages that are not currently in use but are reserved for
    future use.  The percentage at the right is the number of freelist pages
    divided by the total number of pages in the file.

Pages of auto-vacuum overhead

    The number of pages that store data used by the database to facilitate
    auto-vacuum. This is zero for databases that do not support auto-vacuum.

Number of tables in the database

    The number of tables in the database, including the SQLITE_SCHEMA table
    used to store schema information.

Number of indices

    The total number of indices in the database.

Number of defined indices

    The number of indices created using an explicit CREATE INDEX statement.

Number of implied indices

    The number of indices used to implement PRIMARY KEY or UNIQUE constraints
    on tables.

Size of the file in bytes

    The total amount of disk space used by the entire database files.

Bytes of user payload stored

    The total number of bytes of user payload stored in the database. The
    schema information in the SQLITE_SCHEMA table is not counted when
    computing this number.  The percentage at the right shows the payload
    divided by the total file size.

Percentage of total database

    The amount of the complete database file that is devoted to storing
    information described by this category.

Number of entries

    The total number of B-Tree key/value pairs stored under this category.

Bytes of storage consumed

    The total amount of disk space required to store all B-Tree entries
    under this category.  The is the total number of pages used times
    the pages size.

Bytes of payload

    The amount of payload stored under this category.  Payload is the data
    part of table entries and the key part of index entries.  The percentage
    at the right is the bytes of payload divided by the bytes of storage 
    consumed.

Average payload per entry

    The average amount of payload on each entry.  This is just the bytes of
    payload divided by the number of entries.

Average unused bytes per entry

    The average amount of free space remaining on all pages under this
    category on a per-entry basis.  This is the number of unused bytes on
    all pages divided by the number of entries.

Non-sequential pages

    The number of pages in the table or index that are out of sequence.
    Many filesystems are optimized for sequential file access so a small
    number of non-sequential pages might result in faster queries,
    especially for larger database files that do not fit in the disk cache.
    Note that after running VACUUM, the root page of each table or index is
    at the beginning of the database file and all other pages are in a
    separate part of the database file, resulting in a single non-
    sequential page.

Maximum payload per entry

    The largest payload size of any entry.

Entries that use overflow

    The number of entries that user one or more overflow pages.

Total pages used

    This is the number of pages used to hold all information in the current
    category.  This is the sum of index, primary, and overflow pages.

Index pages used

    This is the number of pages in a table B-tree that hold only key (rowid)
    information and no data.

Primary pages used

    This is the number of B-tree pages that hold both key and data.

Overflow pages used

    The total number of overflow pages used for this category.

Unused bytes on index pages

    The total number of bytes of unused space on all index pages.  The
    percentage at the right is the number of unused bytes divided by the
    total number of bytes on index pages.

Unused bytes on primary pages

    The total number of bytes of unused space on all primary pages.  The
    percentage at the right is the number of unused bytes divided by the
    total number of bytes on primary pages.

Unused bytes on overflow pages

    The total number of bytes of unused space on all overflow pages.  The
    percentage at the right is the number of unused bytes divided by the
    total number of bytes on overflow pages.

Unused bytes on all pages

    The total number of bytes of unused space on all primary and overflow 
    pages.  The percentage at the right is the number of unused bytes 
    divided by the total number of bytes.

*******************************************************************************
The entire text of this report can be sourced into any SQL database
engine for further analysis.  All of the text above is an SQL comment.
The data used to generate this report follows:
*/
BEGIN;
CREATE TABLE space_used(
   name clob,        -- Name of a table or index in the database file
   tblname clob,     -- Name of associated table
   is_index boolean, -- TRUE if it is an index, false for a table
   nentry int,       -- Number of entries in the BTree
   leaf_entries int, -- Number of leaf entries
   depth int,        -- Depth of the b-tree
   payload int,      -- Total amount of data stored in this table or index
   ovfl_payload int, -- Total amount of data stored on overflow pages
   ovfl_cnt int,     -- Number of entries that use overflow
   mx_payload int,   -- Maximum payload size
   int_pages int,    -- Number of interior pages used
   leaf_pages int,   -- Number of leaf pages used
   ovfl_pages int,   -- Number of overflow pages used
   int_unused int,   -- Number of unused bytes on interior pages
   leaf_unused int,  -- Number of unused bytes on primary pages
   ovfl_unused int,  -- Number of unused bytes on overflow pages
   gap_cnt int,      -- Number of gaps in the page layout
   compressed_size int  -- Total bytes stored on disk
);
INSERT INTO space_used VALUES('sqlite_schema','sqlite_schema',0,36,36,1,5188,0,0,379,0,1,0,0,27309,0,0,32768);
INSERT INTO space_used VALUES('moz_places','moz_places',0,10955,10894,2,1838131,0,0,1867,1,62,0,32207,121406,0,30,2064384);
INSERT INTO space_used VALUES('moz_historyvisits','moz_historyvisits',0,15884,15873,2,308447,0,0,21,1,12,0,32668,5435,0,8,425984);
INSERT INTO space_used VALUES('moz_inputhistory','moz_inputhistory',0,8,8,1,341,0,0,71,0,1,0,0,32379,0,0,32768);
INSERT INTO space_used VALUES('sqlite_autoindex_moz_inputhistory_1','moz_inputhistory',1,8,8,1,301,0,0,65,0,1,0,0,32435,0,0,32768);
INSERT INTO space_used VALUES('moz_hosts','moz_hosts',0,628,628,1,14640,0,0,49,0,1,0,0,15012,0,0,32768);
INSERT INTO space_used VALUES('sqlite_autoindex_moz_hosts_1','moz_hosts',1,628,628,1,13000,0,0,47,0,1,0,0,17876,0,0,32768);
INSERT INTO space_used VALUES('moz_bookmarks','moz_bookmarks',0,313,313,1,21937,0,0,518,0,1,0,0,9358,0,0,32768);
INSERT INTO space_used VALUES('moz_bookmarks_roots','moz_bookmarks_roots',0,5,5,1,47,0,0,11,0,1,0,0,32693,0,0,32768);
INSERT INTO space_used VALUES('sqlite_autoindex_moz_bookmarks_roots_1','moz_bookmarks_roots',1,5,5,1,47,0,0,11,0,1,0,0,32698,0,0,32768);
INSERT INTO space_used VALUES('moz_keywords','moz_keywords',0,0,0,1,0,0,0,0,0,1,0,0,32760,0,0,32768);
INSERT INTO space_used VALUES('sqlite_autoindex_moz_keywords_1','moz_keywords',1,0,0,1,0,0,0,0,0,1,0,0,32760,0,0,32768);
INSERT INTO space_used VALUES('sqlite_sequence','sqlite_sequence',0,1,1,1,15,0,0,15,0,1,0,0,32741,0,0,32768);
INSERT INTO space_used VALUES('moz_favicons','moz_favicons',0,481,471,2,297630,0,0,7640,1,11,0,32676,59923,0,6,393216);
INSERT INTO space_used VALUES('sqlite_autoindex_moz_favicons_1','moz_favicons',1,471,470,2,35135,0,0,7259,1,2,0,32664,29057,0,1,98304);
INSERT INTO space_used VALUES('moz_anno_attributes','moz_anno_attributes',0,12,12,1,355,0,0,42,0,1,0,0,32357,0,0,32768);
INSERT INTO space_used VALUES('sqlite_autoindex_moz_anno_attributes_1','moz_anno_attributes',1,12,12,1,366,0,0,43,0,1,0,0,32358,0,0,32768);
INSERT INTO space_used VALUES('moz_annos','moz_annos',0,195,195,1,12115,0,0,127,0,1,0,0,19702,0,0,32768);
INSERT INTO space_used VALUES('moz_items_annos','moz_items_annos',0,79,79,1,8649,0,0,384,0,1,0,0,23743,0,0,32768);
INSERT INTO space_used VALUES('sqlite_stat1','sqlite_stat1',0,15,15,1,762,0,0,62,0,1,0,0,31923,0,0,32768);
INSERT INTO space_used VALUES('moz_places_faviconindex','moz_places',1,10894,10891,2,83178,0,0,8,1,4,0,32711,15213,0,3,163840);
INSERT INTO space_used VALUES('moz_places_hostindex','moz_places',1,10894,10885,2,237383,0,0,49,1,10,0,32473,57782,0,7,360448);
INSERT INTO space_used VALUES('moz_places_visitcount','moz_places',1,10894,10891,2,66129,0,0,8,1,4,0,32717,32256,0,3,163840);
INSERT INTO space_used VALUES('moz_places_frecencyindex','moz_places',1,10894,10891,2,76772,0,0,9,1,4,0,32714,21616,0,3,163840);
INSERT INTO space_used VALUES('moz_places_lastvisitdateindex','moz_places',1,10894,10889,2,150784,0,0,14,1,6,0,32651,13179,0,4,229376);
INSERT INTO space_used VALUES('moz_historyvisits_placedateindex','moz_historyvisits',1,15873,15863,2,267273,0,0,17,1,11,0,32516,45668,0,8,393216);
INSERT INTO space_used VALUES('moz_historyvisits_fromindex','moz_historyvisits',1,15873,15869,2,100292,0,0,7,1,5,0,32702,15927,0,4,196608);
INSERT INTO space_used VALUES('moz_historyvisits_dateindex','moz_historyvisits',1,15873,15865,2,206221,0,0,13,1,9,0,32596,41128,0,6,327680);
INSERT INTO space_used VALUES('moz_bookmarks_itemindex','moz_bookmarks',1,313,313,1,2547,0,0,9,0,1,0,0,29274,0,0,32768);
INSERT INTO space_used VALUES('moz_bookmarks_parentindex','moz_bookmarks',1,313,313,1,2393,0,0,9,0,1,0,0,29428,0,0,32768);
INSERT INTO space_used VALUES('moz_bookmarks_itemlastmodifiedindex','moz_bookmarks',1,313,313,1,5020,0,0,17,0,1,0,0,26801,0,0,32768);
INSERT INTO space_used VALUES('moz_places_url_uniqueindex','moz_places',1,10894,10859,2,1010666,0,0,1823,1,36,0,24134,143261,0,16,1212416);
INSERT INTO space_used VALUES('moz_places_guid_uniqueindex','moz_places',1,10894,10887,2,196000,0,0,18,1,8,0,32581,33545,0,5,294912);
INSERT INTO space_used VALUES('moz_bookmarks_guid_uniqueindex','moz_bookmarks',1,313,313,1,5207,0,0,17,0,1,0,0,26614,0,0,32768);
INSERT INTO space_used VALUES('moz_annos_placeattributeindex','moz_annos',1,195,195,1,1871,0,0,10,0,1,0,0,30304,0,0,32768);
INSERT INTO space_used VALUES('moz_items_annos_itemattributeindex','moz_items_annos',1,79,79,1,562,0,0,9,0,1,0,0,31961,0,0,32768);
INSERT INTO space_used VALUES('moz_keywords_placepostdata_uniqueindex','moz_keywords',1,0,0,1,0,0,0,0,0,1,0,0,32760,0,0,32768);
COMMIT;

This page last modified on  2022-01-08 05:02:57 UTC


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 1. Overview
c5: quirks.html#overview
c4:
The SQL language is a "standard".
Even so, no two SQL database engines work exactly alike.
Every SQL implementation has it own peculiarities and oddities,
and SQLite is no exception to this rule.


This document strives to highlight the principal differences
between SQLite and other SQL implementations, as an aid to developers
that are porting to or from SQLite or who are trying to build a
system that works across multiple database engines.


If you are an SQLite user whose has stumbled over some quirk of
SQLite that is not mentioned here, please let the developers know
by posting a brief message on the
SQLite Forum.


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 2. SQLite Is Embedded, Not Client-Server
c5: quirks.html#sqlite_is_embedded_not_client_server
c4:
Whenever comparing SQLite to other SQL database engines like
SQL Server, PostgreSQL, MySQL, or Oracle, it is important first of all
to realize that SQLite is not intended as a replacement or competitor to
any of those systems.  SQLite is serverless.  There is no separate
server process that manages the database.  An application interacts
with the database engine using function calls, not by sending messages
to a separate process or thread.


The fact that SQLite is embedded and serverless instead of being
client/server is a feature, not a bug.


Client/server databases like MySQL, PostgreSQL, SQL Server, Oracle, and
others are an important component of modern systems.
These systems solve an important problem.
But SQLite solves a different problem.
Both SQLite and client/server databases have their role.
Developers who are comparing SQLite against other SQL database engines need
to clearly understand this distinction.


See the Appropriate Uses For SQLite document for additional information.


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 3. Flexible Typing
c5: quirks.html#flexible_typing
c4:
SQLite is flexible with regard to datatypes.  Datatypes are advisory
rather than mandatory.


Some commentators say that SQLite is "weakly typed" and that other
SQL databases are "strongly typed".  We consider these terms to be
inaccurate and even pejorative.  We prefer to say that SQLite is
"flexibly typed" and that other SQL database engines are
"rigidly typed".


See the Datatypes in SQLite document for a detailed
discussion of the type system in SQLite.


The key point is that SQLite is very forgiving of the type of data that
you put into the database.  For example, if a column has a datatype of
"INTEGER" and the application inserts a text string into that column,
SQLite will first try to convert the text string into an integer, just like
every other SQL database engine.  Thus, if one inserts '1234' into
an INTEGER column, that value is converted into an integer 1234 and stored.
But, if you insert a non-numeric string like 'wxyz' into an INTEGER
column, unlike other SQL databases, SQLite does not throw an error.  Instead,
SQLite stores the actual string value in the column.


Similarly, SQLite allows you to store a 2000-character string into a
column of type VARCHAR(50).  Other SQL implementations would either throw
an error or truncate the string.  SQLite stores the entire 2000-character
string with no loss of information and without complaint.


Where this ends up causing problems is when developers do some initial
coding work using SQLite and get their application working, but then try
to convert to another database like PostgreSQL or SQL Server for deployment.
If the application is initially taking advantage of SQLite's flexible typing,
then it will fail when moved to another database that uses a more rigid
and unforgiving type enforcement policy.


Flexible typing is a feature of SQLite, not a bug.  Flexible typing
is about freedom.
Nevertheless, we recognize that this feature does sometimes cause
confusion for developers who are accustomed to working with
other databases that are more strict with regard to data type rules.
In retrospect, perhaps it would have been less confusing if SQLite had merely
implemented an ANY datatype so that developers could explicitly state
when they wanted to use flexible typing, rather than making flexible
typing the default.
As an accommodation for those who expect rigid typing, SQLite version 3.37.0
(2021-11-27) introduced the option of STRICT tables.
These either impose
the mandatory datatype constraints found in other SQL database engines,
or allow the explicit ANY datatype to retain SQLite's flexible typing.


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 3.1. No Separate BOOLEAN Datatype
c5: quirks.html#no_separate_boolean_datatype
c4:
Unlike most other SQL implementations,
SQLite does not have a separate BOOLEAN data type.
Instead, TRUE and FALSE are (normally) represented as integers 1 and 0,
respectively.
This does not seem to cause many problems, as we seldom get complaints
about it.  But it is important to recognize.


Beginning with SQLite version 3.23.0 (2018-04-02), SQLite also
recognizes TRUE and FALSE keywords as aliases for integer values 1 and 0,
respectively.
This provides better compatibility with other SQL implementations.
But for backwards compatibility, if there are columns named TRUE or
FALSE, then the keywords are treated as identifiers referencing those
columns, rather than BOOLEAN literals.


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 3.2. No Separate DATETIME Datatype
c5: quirks.html#no_separate_datetime_datatype
c4:
SQLite has no DATETIME datatype.
Instead, dates and times can be stored in any of these ways:


 As a TEXT string in the ISO-8601 format.  Example: '2018-04-02 12:13:46'.
 As an INTEGER number of seconds since 1970 (also known as "unix time").
 As a REAL value that is the fractional
     Julian day number.



The built-in date and time functions of SQLite understand date/times in
all of the formats above, and can freely change between them.
Which format you use, is entirely up to your application.


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 3.3. The datatype is optional
c5: quirks.html#the_datatype_is_optional
c4:
Because SQLite is flexible and forgiving with regard to datatypes,
table columns can be created that have no specified datatype at all.
For example:
CREATE TABLE t1(a,b,c,d);

The table "t1" has four columns "a", "b", "c", and "d" that have
no particular datatype assigned.  You can store anything you want in
any of those columns.


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 4. Foreign Key Enforcement Is Off By Default
c5: quirks.html#foreign_key_enforcement_is_off_by_default
c4:
SQLite has parsed foreign key constraints for time out of mind,
but added the ability to actually enforce those constraints much later,
with version 3.6.19 (2009-10-14).  By the time foreign key constraint
enforcement was added, there were already countless millions of databases
in circulation that contained foreign key constraints, some of which
were not correct.  To avoid breaking those legacy databases, foreign key
constraint enforcement is turned off by default in SQLite.

Applications can activate foreign key enforcement at run-time using
the PRAGMA foreign_keys statement.  Or, foreign key enforcement can
be activated at compile-time using the
-DSQLITE_DEFAULT_FOREIGN_KEYS=1 compile-time option.


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 5. PRIMARY KEYs Can Sometimes Contain NULLs
c5: quirks.html#primary_keys_can_sometimes_contain_nulls
c4:
Usually (the exceptions are INTEGER PRIMARY KEY tables and
WITHOUT ROWID tables) a PRIMARY KEY in an SQLite table is really
the same as a UNIQUE constraint.  Due to an historical oversight,
the column values of such a PRIMARY KEY are allowed to be NULL.
This is a bug, but by the time the problem was discovered there
where so many databases in circulation that depended on the bug that
the decision was made to support the buggy behavior moving forward.

The value of an INTEGER PRIMARY KEY column must always be a
non-NULL integer.  The PRIMARY KEY columns of a WITHOUT ROWID
table are also required to be non-NULL.


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 6. Aggregate Queries Can Contain Non-Aggregate Result Columns That Are Not In The GROUP BY Clause
c4:
In most SQL implementations, output columns of an aggregate query
may only reference aggregate functions or columns named in the
GROUP BY clause.  It does not make good sense to reference an ordinary
column in an aggregate query because each output row might be composed
from two or more rows in the input table(s).


SQLite does not enforce this restriction.
The output columns from an aggregate query can be arbitrary expressions
that include columns not found in GROUP BY clause.
This feature has two uses:



With SQLite (but not any other SQL implementation that we know of) if
an aggregate query contains a single min() or max() function, then the
values of columns used in the output are taken from the row where
the min() or max() value was achieved.  If two or more rows have the
same min() or max() value, then the columns values will be chosen arbitrarily
from one of those rows.

For example to find the highest paid employee:
SELECT max(salary), first_name, last_name FROM employee;


In the query above, the values for the first_name and last_name columns
will correspond to the row that satisfied the max(salary) condition.


If a query contains no aggregate functions at all, then a GROUP BY
clause can be added as a substitute for the DISTINCT ON clause.  In other words,
output rows are filtered so that only one row is shown for each distinct
set of values in the GROUP BY clause.  If two or more output rows would
have otherwise had the same set of values for the GROUP BY columns, then
one of the rows is chosen arbitrarily.  (SQLite supports DISTINCT but not
DISTINCT ON, whose functionality is provided instead by GROUP BY.)

c5:
quirks.html#aggregate_queries_can_contain_non_aggregate_result_columns_that_are_not_in_the_group_by_clause


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 7. Does Not Do Full Unicode Case Folding By Default
c5: quirks.html#does_not_do_full_unicode_case_folding_by_default
c4:
SQLite does not know about the upper-case/lower-case distinction
for all unicode characters.  SQL functions like
upper() and lower() only work on ASCII characters.  There are two
reasons for this:

 Though stable now, when SQLite was first designed, the rules for
     unicode case folding were still in flux.  That means that the
     behavior might have changed with each new unicode release, disrupting
     applications and corrupting indexes in the process.
 The tables necessary to do full and proper unicode case folding are
     larger than the whole SQLite library.


Full unicode case folding is supported in SQLite if it is compiled
with the -DSQLITE_ENABLE_ICU option and linked against the
International Components for Unicode
library.


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 8. Double-quoted String Literals Are Accepted
c5: quirks.html#double_quoted_string_literals_are_accepted
c4:
The SQL standard requires double-quotes around identifiers
and single-quotes around string literals.  For example:

 "this is a legal SQL column name"
 'this is an SQL string literal'


SQLite accepts both of the above.  But, in an effort to be compatible
with MySQL 3.x (which was one of the most widely used RDBMSes
when SQLite was first being designed) SQLite will also interpret
a double-quotes string as
string literal if it does not match any valid identifier.

This misfeature means that a misspelled double-quoted
identifier will be interpreted as a string literal, rather than generating
an error.
It also lures developers who are new to the SQL language into the
bad habit of using double-quoted string literals when they
really need to learn to use the correct single-quoted string literal form.

In hindsight, we should not have tried to make SQLite accept MySQL 3.x
syntax, and should have never allowed double-quoted string literals.
However, there are countless applications that make use of
double-quoted string literals and so we continue to support
that capability to avoid breaking legacy.

As of SQLite 3.27.0 (2019-02-07) the use of a double-quoted
string literal causes a warning message to be sent to the error log.

As of SQLite 3.29.0 (2019-07-10) the use of double-quoted
string literals can be disabled at run-time using the
SQLITE_DBCONFIG_DQS_DDL and SQLITE_DBCONFIG_DQS_DML actions
to sqlite3_db_config().  The default settings can be altered
at compile-time using the -DSQLITE_DQS=N compile-time
option.  Application developers are encouraged to compile using
-DSQLITE_DQS=0 in order to disable the double-quoted string literal
misfeature by default.  If that is not possible, then disable
double-quoted string literals for individual database connections
using C-code like this:

sqlite3_db_config(db, SQLITE_DBCONFIG_DQS_DDL, 0, (void*)0);
sqlite3_db_config(db, SQLITE_DBCONFIG_DQS_DML, 0, (void*)0);

Or, if double-quoted string literals are disabled by default, but need
to be selectively enabled for some historical database connections,
that can be done using the same C-code as shown above except with the
third parameter changed from 0 to 1.

As of SQLite 3.41.0 (2023-02-21) SQLITE_DBCONFIG_DQS_DDL and
SQLTIE_DBCONFIG_DQS_DML are disabled by default in the CLI.  Use
the ".dbconfig" dot-command to reenable the legacy behavior if
desired.


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 9. Keywords Can Often Be Used As Identifiers
c5: quirks.html#keywords_can_often_be_used_as_identifiers
c4:
The SQL language is rich in keywords.
Most SQL implementations do not allow keywords to be used as identifiers
(names of tables or columns) unless they are enclosed in double-quotes.
But SQLite is more flexible.  Many keywords can be used as identifiers without
needing to be quoted, as long as those keywords are used in a context where
it is clear that they are intended to be an identifier.

For example, the following statement is valid in SQLite:
CREATE TABLE union(true INT, with BOOLEAN);


The same SQL statement will fail on every other SQL implementation that
we know of due to the use of keywords "union", "true", and "with" as
identifiers.

The ability to use keywords as identifiers promotes backwards compatibility.
As new keywords are added, legacy schemas that just happen to use those
keywords as table or column names continue to work.  However, the ability
to use a keyword as an identifier sometimes leads to surprising outcomes.
For example:
CREATE TRIGGER AFTER INSERT ON tableX BEGIN
  INSERT INTO tableY(b) VALUES(new.a);
END;

The trigger created by the previous statement is named "AFTER"
and it is a "BEFORE" trigger.  The "AFTER" token is used as an identifier
instead of as a keyword, as that is the only way to parse the statement.
Another example:
CREATE TABLE tableZ(INTEGER PRIMARY KEY);

The tableZ table has a single column named "INTEGER".  That column
has no datatype specified, but it is the PRIMARY KEY.
The column is not the INTEGER PRIMARY KEY for the table because
it has no datatype.  The "INTEGER" token is used as an identifier for the
column name, not as a datatype keyword.


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 10. Dubious SQL Is Allowed Without Any Error Or Warning
c5: quirks.html#dubious_sql_is_allowed_without_any_error_or_warning
c4:
The original implementation of SQLite sought to follow
Postel's Law which
states in part "Be liberal in what you accept".
This used to be considered good design - that a system would accept
dodgy inputs and try to do the best it could without complaining too much.
But lately, people have come to realize that it is sometimes better to
be strict in what you accept, so as to more easily find errors in the
input.


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 11. AUTOINCREMENT Does Not Work The Same As MySQL
c5: quirks.html#autoincrement_does_not_work_the_same_as_mysql
c4:
The AUTOINCREMENT feature in SQLite works differently than
it does in MySQL.  This often causes confusion for people who
initially learned SQL on MySQL and then start using SQLite, and
expect the two systems to work identically.

See the SQLite AUTOINCREMENT documentation for
detailed instructions on what AUTOINCREMENT does and does not do
in SQLite.


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 12. NUL Characters Are Allowed In Text Strings
c5: quirks.html#nul_characters_are_allowed_in_text_strings
c4:
NUL characters (ASCII code 0x00 and Unicode \u0000) may appear in
the middle of strings in SQLite.  This can lead to unexpected behavior.
See the "NUL characters in strings" document for further information.


c2: Quirks, Caveats, and Gotchas In SQLite
c3: 13. SQLite Distinguishes Between Integer And Text Literals
c5: quirks.html#sqlite_distinguishes_between_integer_and_text_literals
c4:
SQLite says that the following query returns false:
SELECT 1='1';

It does this because an integer is not a string.  
Every other major SQL database engine says this is true, for reasons
that the creator of SQLite does not understand.
This page last modified on  2023-04-16 12:53:27 UTC


c2: sqldiff.exe: Database Difference Utility
c3: 1. Usage
c5: sqldiff.html#usage
c4:
The sqldiff.exe binary is a command-line utility program that
displays content differences between SQLite databases.  Example
usage:

sqldiff &#91;options&#93; database1.sqlite database2.sqlite



The usual output is an SQL script that will transform
database1.sqlite (the "source" database) into database2.sqlite
(the "destination" database).  This behavior can be
altered using command-line switches:


--changeset FILE
Do not write changes to standard output.  Instead, write a (binary)
       changeset file into FILE.  The changeset can be interpreted using
       the session extension to SQLite.
--lib LIBRARY
-L LIBRARY
Load the shared library or DLL file LIBRARY into SQLite prior to
       computing the differences.  This can be used to add application-defined
       collating sequences that are required by the schema.
--primarykey
Use the schema-defined PRIMARY KEY instead of the rowid to
       pair rows in the source and destination database.  (See additional
       explanation below.)
--schema
Show only column name and table differences in the schema,
       not the table content
--summary
Show how many rows have changed on each table, but do not show
       the actual changes
--table TABLE
Show only the differences in content for TABLE, not for the
       entire database
--transaction
Wrap SQL output in a single large transaction
--vtab
Add support for handling FTS3, FTS5 and rtree virtual tables.
       See below for details.


c2: sqldiff.exe: Database Difference Utility
c3: 2. How It Works
c5: sqldiff.html#how_it_works
c4:
The sqldiff.exe utility works by finding rows in the source and
destination that are logical "pairs".  The default behavior is to
treat two rows as pairs if they are in tables with the same name
and they have the same rowid, or in the case of a WITHOUT ROWID
table if they have the same PRIMARY KEY.  Any differences in the
content of paired rows are output as UPDATEs.  Rows in the source
database that could not be paired are output as DELETEs.  Rows in
the destination database that could not be paired are output as
INSERTs.

The --primarykey flag changes the pairing algorithm slightly so
that the schema-declared PRIMARY KEY is always used for pairing,
even on tables that have a rowid.  This is often a better choice
for finding differences, however it can lead to missed differences in
the case of rows that have one or more PRIMARY KEY columns set to
NULL.


c2: sqldiff.exe: Database Difference Utility
c3: 3. Limitations
c5: sqldiff.html#limitations
c4:
The sqldiff.exe utility does not compute changesets for
either: rowid tables for which the rowid is inaccessible;
or tables which have no explicit primary key.
Given the --changeset option, sqldiff omits them from the comparison.
Examples of such tables are:

CREATE TABLE NilChangeset (
   -- inaccessible rowid due to hiding its aliases
   "rowid" TEXT,
   "oid" TEXT,
   "_rowid_" TEXT
);


and

CREATE TABLE NilChangeset (
   -- no explicit primary key
   "authorId" TEXT,
   "bookId" TEXT
);



When sqldiff is made to compare only such tables, no error occurs.
However, the result may be unexpected.
For example, the effect of this invocation:

sqldiff --changeset CHANGESET_OUT --table NilChangeset db1.sdb db2.sdb


will be to produce an empty file named "CHANGESET_OUT". See  session limitations for details.

The sqldiff.exe utility does not (currently) display differences in
TRIGGERs or VIEWs.


The sqldiff utility is not designed to support schema migrations
and is forgiving with respect to differing column definitions.
Normally, only the column names and their order are compared
for like-named tables before content comparison proceeds.

However, the single-table comparison option, with "sqlite_schema"
named, can be used to show or detect detailed schema differences
between a pair of databases.
When doing this, the output should not be used directly to modify a database.




By default, differences in the schema or content of virtual tables are
not reported on.

However, if a virtual table implementation creates real tables (sometimes
referred to as "shadow" tables) within the database to store its data in, then
sqldiff.exe does calculate the difference between these. This can have
surprising effects if the resulting SQL script is then run on a database that
is not exactly the same as the source database. For several of SQLite's
bundled virtual tables (FTS3, FTS5, rtree and others), the surprising effects
may include corruption of the virtual table content.

 If the --vtab option is passed to sqldiff.exe, then it ignores all
underlying shadow tables belonging to an FTS3, FTS5 or rtree virtual table
and instead includes the virtual table differences directly.


This page last modified on  2023-01-06 00:45:39 UTC


c2: Maintaining Private Branches Of SQLite
c3: 1. Introduction
c5: privatebranch.html#_introduction
c4:
SQLite is designed to meet most developer's needs without any 
changes or customization.  When changes are needed, they can normally
be accomplished using start-time (1)
or runtime
(2)
(3)
(4) configuration methods
or via compile-time options.  It is very rare that an application 
developer will need to edit the SQLite source code in order to 
incorporate SQLite into a product.

We call custom modifications to the SQLite source code that are held
for the use of a single application a "private branch".  When a private
branch becomes necessary, the application developer must take on the
task of keeping the private branch in synchronization with the public
SQLite sources.  This is tedious.  It can also be tricky, since while
the SQLite file format and published interfaces are very stable, the
internal implementation of SQLite changes quite rapidly.  Hundreds or
thousands of lines of code might change for any given SQLite point release.


This article outlines one possible method for keeping a private branch
of SQLite in sync with the public SQLite source code.
There are many ways of maintaining a private branch, of course.
Nobody is compelled to use the method describe here.
This article is not trying to impose a particular procedure on 
maintainers of private branches.  The point of this article is to offer
an example of one process for maintaining a private branch which can
be used as a template for designing processes best suited for the
circumstances of each individual project.


c2: Maintaining Private Branches Of SQLite
c3: 2. The Basic Idea
c5: privatebranch.html#_the_basic_idea
c4:
We propose to use the
fossil software configuration management
system to set up two branches.  One branch (the "public branch" or "trunk")
contains the published SQLite sources and the other branch is the 
private branch which contains the code that is customized for the project.
Whenever a new public release of SQLite is made, that release is added to
the public branch and then the changes are merged into the private branch.

This document proposes to use
fossil,
but any other distributed software configuration management system such as
monotone or
mercurial (a.k.a. "hg"), or
git could serve just as well.
The concept will be the same, 
though the specifics of the procedure will vary.

The diagram at the right illustrates the concept.
One begins with a standard SQLite release.  For the
sake of example, suppose that one intends to create a
private branch off of SQLite version 3.6.15.  In the
diagram this is version (1).  The
maintainer makes an exact copy of the baseline
SQLite into the branch space, shown as version (2).
Note that (1) and (2) are exactly the same.  Then
the maintainer applies the private changes to 
version (2) resulting in version (3).  In other words,
version (3) is SQLite version 3.6.15 plus edits.

Later, SQLite version 3.6.16 is released, as shown 
by circle (4) in the diagram.  At the point, the private
branch maintainer does a merge which takes all of the
changes going from (1) to (4) and applies those changes to
(3).  The result is version (5), which is SQLite 3.6.16
plus edits.

There might be merge conflicts.  In other words, it might
be that the changes from (2) to (3) are incompatible with the
changes from (1) to (4).  In that case, the maintainer will
have to manually resolve the conflicts.  Hopefully conflicts
will not come up that often.  Conflicts are less likely to
occur when the private edits are kept to a minimum.

The cycle above can be repeated many times.  The
diagram shows a third SQLite release, 3.6.17 in
circle (6).  The private branch maintainer can do
another merge in order to incorporate the changes
moving from (4) to (6) into the private branch, resulting
in version (7).


c2: Maintaining Private Branches Of SQLite
c3: 3. The Procedure
c5: privatebranch.html#_the_procedure
c4:
The remainder of this document will guide the reader through
the steps needed to maintain a private branch.  The general idea
is the same as outlined above.  This section merely provides more
detail.

We emphasize again that these steps are not intended to be the only
acceptable method for maintaining private branch.  This approach
is one of many.  Use this document as a baseline for preparing
project-specific procedures.  Do not be afraid to experiment.


c2: Maintaining Private Branches Of SQLite
c3: 3.1. Obtain The Software
c5: privatebranch.html#_obtain_the_software
c4:
Fossil is a computer program
that must be installed on your machine before you use it.
Fortunately, installing fossil is very easy.  Fossil is a single
"*.exe" file that you simply download and run.  To uninstall fossil,
simply delete the exe file.  
Detailed instructions for installing and getting started with
fossil are available on the 
fossil website.


c2: Maintaining Private Branches Of SQLite
c3: 3.2. Create A Project Repository
c5: privatebranch.html#_create_a_project_repository
c4:
Create a fossil repository to host the private branch using the
following command:


fossil new private-project.fossil


You can call your project anything you like.  The ".fossil"
suffix is optional.  For this document, we will continue to call the
project "private-project.fossil".  Note that
private-project.fossil is an ordinary disk file (actually an
SQLite database) that will contain your complete project history.
You can make a backup of the project simply by making a copy of that
one file.

If you want to configure the new project, type:


fossil ui private-project.fossil


The "ui" command will cause fossil to run a miniature built-in webserver
and to launch your web-browser pointing
at that webserver.  You can use your web-browser to configure your project
in various ways.  See the instructions on the fossil website for additional
information.

Once the project repository is created, create an open checkout of the
project by moving to the directory where you want to keep all of the
project source code and typing:


fossil open private-project.fossil


You can have multiple checkouts of the same project if you want. 
And you can "clone" the repository to different machines so that multiple
developers can use it.  See the fossil website for further information.


c2: Maintaining Private Branches Of SQLite
c3: 3.3. Installing The SQLite Baseline In Fossil
c5: privatebranch.html#_installing_the_sqlite_baseline_in_fossil
c4:
The repository created in the previous step is initially empty.  The
next step is to load the baseline SQLite release - circle (1) in the diagram
above.

Begin by obtaining a copy of SQLite in whatever form you use it.
The public SQLite you obtain should be as close to your private edited
copy as possible.  If your project uses the SQLite amalgamation, then
get a copy of the amalgamation.  If you use the preprocessed separate
source files, get those instead.  Put all the source files in the
checkout directory created in the previous step.

The source code in public SQLite releases uses unix line endings
(ASCII code 10: "newline" only, NL) and spaces instead of tabs.  If you will
be changing the line ending to windows-style line endings
(ASCII codes 13, 10: "carriage-return" and "newline"; CR-NL) or if you will be
changing space indents into tab indents, make that change now
before you check in the baseline.  The merging process will only work
well if the differences between the public and the private branches are
minimal.  If every single line of the source file is changed in the
private branch because you changed from NL to CR-NL line endings, then
the merge steps will not work correctly.

Let us assume that you are using the amalgamation source code.
Add the baseline to your project as follows:


fossil add sqlite3.c sqlite3.h


If you are using separate source files, name all of the source files instead
of just the two amalgamation source files.  Once this is done, commit your
changes as follows:


fossil commit


You will be prompted for a check-in comment.  Say whatever you like.
After the commit completes, your baseline will be part of the repository.
The following command, if you like, to see this on the "timeline":



fossil ui


That last command is the same "ui" command that we ran before.  It
starts a mini-webserver running and points your web browser at it.  But
this time we didn't have to specify the repository file because we are
located inside a checkout and so fossil can figure out the repository for
itself.  If you want to type in the repository filename as the second
argument, you can.  But it is optional.

If you do not want to use your web browser to view the new check-in,
you can get some information from the command-line using commands like
these:


fossil timeline
fossil info
fossil status


c2: Maintaining Private Branches Of SQLite
c3: 3.4. Creating The Private Branch
c5: privatebranch.html#_creating_the_private_branch
c4:
The previous step created circle (1) in the diagram above.
This step will create circle (2).  Run the following command:

 
fossil branch new private trunk -bgcolor "#add8e8"


This command will create a new branch named "private" (you can use
a different name if you like) and assign it a background color
of light blue ("#add8e8").  You can omit the background color if you want,
though having a distinct background does make it easier to tell the
branch from the "trunk" (the public branch) on timeline displays.  You
can change the background color of the private branch or of the public
branch (the "trunk") using the web interface if you like.

The command above created the new branch.  But your checkout is
still on the trunk - a fact you can see by running the command:


fossil info


To change your check-out to the private branch, type:


fossil update private


You can run the "info" command again to verify that you are on the
private branch.  To go back to the public branch, type:


fossil update trunk


Normally, fossil will modify all the files in your checkout when switching
between the private and the public branches.  But at this point, the files
are identical in both branches so no modifications need to be made.


c2: Maintaining Private Branches Of SQLite
c3: 3.5. Adding Customizations To The Code In The Private Branch
c5: privatebranch.html#_adding_customizations_to_the_code_in_the_private_branch
c4:
Now it is time to make the private, custom modifications to SQLite
which are the whole point of this exercise.  Switch to the private branch
(if you are not already there) using the "fossil update private"
command, then bring up the source files in your text editor and make
whatever changes you want to make.  Once you have finished making
changes, commit those changes using this command:


fossil commit


You will be prompted once again to enter a commit describing your
changes.  Then the commit will occur.  The commit creates a new checkin
in the repository that corresponds to circle (3) in the diagram above.

Now that the public and private branches are different, you can run
the "fossil update trunk" and "fossil update private"
commands and see that fossil really does change the files in the checkout
as you switch back and forth between branches.

Note that in the diagram above, we showed the private edits as a single
commit.  This was for clarity of presentation only.  There is nothing to stop
you from doing dozens or hundreds of separate tiny changes and committing
each separately.  In fact, making many small changes is the preferred way
to work.  The only reason for doing all the changes in a single commit
is that it makes the diagram easier to draw.


c2: Maintaining Private Branches Of SQLite
c3: 3.6. Incorporating New Public SQLite Releases
c5: privatebranch.html#_incorporating_new_public_sqlite_releases
c4:
Suppose that after a while (about a month, usually) a new version of
SQLite is released: 3.6.16.  You will want to incorporate this new
public version of SQLite into your repository in the public branch (the
trunk).  To do this, first change your repository over to the trunk:


fossil update trunk


Then download the new version of the SQLite sources and overwrite the
files that are in the checkout.

If you made NL to CR-NL line ending changes or space to tab
indentation changes in the original baseline, make the same changes
to the new source file.

Once everything is ready, run the "fossil commit" command to
check in the changes.  This creates circle (4) in the diagram above.


c2: Maintaining Private Branches Of SQLite
c3: 3.7. Merging Public SQLite Updates Into The Private Branch
c5: privatebranch.html#_merging_public_sqlite_updates_into_the_private_branch
c4:
The next step is to move the changes in the public branch over into
the private branch.  In other words, we want to create circle (5) in the
diagram above.  Begin by changing to the private branch using
"fossil update private".  Then type this command:


fossil merge trunk


The "merge" command attempts to apply all the changes between
circles (1) and (4) to the files in the local checkout.  Note that
circle (5) has not been created yet.  You will need to run the
"commit" to create circle (5).

It might be that there are conflicts in the merge.  Conflicts
occur when the same line of code was changed in different ways between
circles (1) and (4) versus circles (2) and (3).  The merge command will
announce any conflicts and will include both versions of the conflicting
lines in the output.  You will need to bring up the files that contain
conflicts and manually resolve the conflicts.

After resolving conflicts, many users like to compile and test the
new version before committing it to the repository.  Or you can commit
first and test later.  Either way, run the "fossil commit"
command to check-in the circle (5) version.


c2: Maintaining Private Branches Of SQLite
c3: 3.8. Further Updates
c5: privatebranch.html#_further_updates
c4:
As new versions of SQLite are released, repeat steps 3.6 and 3.7 to
add changes in the new release to the private branch.
Additional private changes can be
made on the private branch in between releases if desired.


c2: Maintaining Private Branches Of SQLite
c3: 4. Variations
c5: privatebranch.html#_variations
c4:
Since this document was first written, the canonical SQLite source code
has been moved from the venerable CVS system into a Fossil repository at
https://www.sqlite.org/src.  This means that if you are working with
canonical SQLite source code (as opposed to the amalgamation source code
files, sqlite3.c and sqlite3.h) then you can create a private repository
simply by cloning the official repository:


fossil clone https://www.sqlite.org/src private-project.fossil


This command both creates the new repository and populates it with
all the latest SQLite code.  You can then create a private branch as
described in section 3.4.

When the private repository is created by cloning, incorporating new
public SQLite releases becomes much easier too.  To pull in all of the
latest changes from the public SQLite repository, simply move into
the open check-out and do:


fossil update


Then continue to merge the changes in "trunk" with your "private"
changes as described in section 3.7.


c2: Compile-time Options
c3: 1. Overview
c5: compile.html#overview
c4:
For most purposes, SQLite can be built just fine using the default
compilation options. However, if required, the compile-time options
documented below can be used to
omit SQLite features (resulting in
a smaller compiled library size) or to change the
default values of some parameters.



Every effort has been made to ensure that the various combinations
of compilation options work harmoniously and produce a working library.
Nevertheless, it is strongly recommended that the SQLite test-suite
be executed to check for errors before using an SQLite library built
with non-standard compilation options.


c2: Compile-time Options
c3: 2. Recommended Compile-time Options
c5: compile.html#recommended_compile_time_options
c4:
The following compile-time options are recommended for applications that
are able to use them, in order to minimized the number of CPU cycles and
the bytes of memory used by SQLite.
Not all of these compile-time options are usable by every application.
For example, the SQLITE_THREADSAFE=0 option is only usable by applications
that never access SQLite from more than one thread at a time.  And the
SQLITE_OMIT_PROGRESS_CALLBACK option is only usable by applications that
do not use the sqlite3_progress_handler() interface.  And so forth.

It is impossible to test every possible combination of compile-time
options for SQLite.  But the following set of compile-time options is
one configuration that is always fully tested.


SQLITE_DQS=0.
This setting disables the double-quoted string literal misfeature.

SQLITE_THREADSAFE=0.
Setting -DSQLITE_THREADSAFE=0 causes all of the mutex and thread-safety logic
in SQLite to be omitted.  This is the single compile-time option causes SQLite
to run about 2% faster and also reduces the size of the library by about 2%.
But the downside is that using the compile-time option means that SQLite can never
be used by more than a single thread at a time, even if each thread has its own
database connection.

SQLITE_DEFAULT_MEMSTATUS=0.
This setting causes the sqlite3_status() interfaces that track memory usage
to be disabled.  This helps the sqlite3_malloc() routines run much faster,
and since SQLite uses sqlite3_malloc() internally, this helps to make the
entire library faster.

SQLITE_DEFAULT_WAL_SYNCHRONOUS=1.
For maximum database safety following a power loss, the setting of
PRAGMA synchronous=FULL is recommended.  However, in WAL mode, complete
database integrity is guaranteed with PRAGMA synchronous=NORMAL.  With
PRAGMA synchronous=NORMAL in WAL mode, recent changes to the database might
be rolled back by a power loss, but the database will not be corrupted.
Furthermore, transaction commit is much faster in WAL mode using
synchronous=NORMAL than with the default synchronous=FULL.  For these
reasons, it is recommended that the synchronous setting be changed from
FULL to NORMAL when switching to WAL mode.  This compile-time option will
accomplish that.

SQLITE_LIKE_DOESNT_MATCH_BLOBS.
Historically, SQLite has allowed BLOB operands to the LIKE and GLOB
operators.  But having a BLOB as an operand of LIKE or GLOB complicates
and slows the LIKE optimization.  When this option is set, it means that
the LIKE and GLOB operators always return FALSE if either operand is a BLOB.
That simplifies the implementation of the LIKE optimization and allows
queries that use the LIKE optimization to run faster.

SQLITE_MAX_EXPR_DEPTH=0.
Setting the maximum expression parse-tree depth to zero disables all checking
of the expression parse-tree depth, which simplifies the code resulting in
faster execution, and helps the parse tree to use less memory.

SQLITE_OMIT_DECLTYPE.
By omitting the (seldom-needed) ability to return the declared type of
columns from the result set of query, prepared statements can be made
to consume less memory.

SQLITE_OMIT_DEPRECATED.
Omitting deprecated interfaces and features will not help SQLite to
run any faster.  It will reduce the library footprint, however.  And
it is the right thing to do.

SQLITE_OMIT_PROGRESS_CALLBACK.
The progress handler callback counter must be checked in the inner loop
of the bytecode engine.  By omitting this interface, a single conditional
is removed from the inner loop of the bytecode engine, helping SQL statements
to run slightly faster.

SQLITE_OMIT_SHARED_CACHE.
Omitting the possibility of using shared cache allows many conditionals
in performance-critical sections of the code to be eliminated.  This can
give a noticeable improvement in performance.

SQLITE_USE_ALLOCA.
Make use of alloca() for dynamically allocating temporary stack space for
use within a single function, on systems that support alloca().  Without
this option, temporary space is allocated from the heap.

SQLITE_OMIT_AUTOINIT.
The SQLite library needs to be initialized using a call to
sqlite3_initialize() before certain interfaces are used.
This initialization normally happens automatically the first time
it is needed.  However, with the SQLITE_OMIT_AUTOINIT option, the automatic
initialization is omitted.  This helps many API calls to run a little faster
(since they do not have to check to see if initialization has already occurred
and then run initialization if it has not previously been invoked) but it
also means that the application must call sqlite3_initialize() manually.
If SQLite is compiled with -DSQLITE_OMIT_AUTOINIT and a routine like
sqlite3_malloc() or sqlite3_vfs_find() or sqlite3_open() is invoked
without first calling sqlite3_initialize(), the likely result will be
a segfault.


When all of the recommended compile-time options above are used,
the SQLite library will be approximately 3% smaller and use about 5% fewer
CPU cycles.  So these options do not make a huge difference.  But in
some design situations, every little bit helps.


c2: Compile-time Options
c3: 3. Platform Configuration
c5: compile.html#_platform_configuration
c4:
_HAVE_SQLITE_CONFIG_H
  If the _HAVE_SQLITE_CONFIG_H macro is defined
  then the SQLite source code will attempt to #include a file named "config.h".
  The "config.h" file usually contains other configuration options, especially
  "HAVE_INTERFACE" type options generated by autoconf scripts.

HAVE_FDATASYNC
  If the HAVE_FDATASYNC compile-time option is true, then the default VFS
  for unix systems will attempt to use fdatasync() instead of fsync() where
  appropriate.  If this flag is missing or false, then fsync() is always used.

HAVE_GMTIME_R
  If the HAVE_GMTIME_R option is true and if SQLITE_OMIT_DATETIME_FUNCS is true,
  then the CURRENT_TIME, CURRENT_DATE, and CURRENT_TIMESTAMP keywords will use
  the threadsafe "gmtime_r()" interface rather than "gmtime()".  In the usual case
  where SQLITE_OMIT_DATETIME_FUNCS is not defined or is false, then the
  built-in date and time functions are used to implement the CURRENT_TIME,
  CURRENT_DATE, and CURRENT_TIMESTAMP keywords and neither gmtime_r() nor
  gmtime() is ever called.

HAVE_ISNAN
  If the HAVE_ISNAN option is true, then SQLite invokes the system library isnan()
  function to determine if a double-precision floating point value is a NaN.
  If HAVE_ISNAN is undefined or false, then SQLite substitutes its own home-grown
  implementation of isnan().

HAVE_LOCALTIME_R
  If the HAVE_LOCALTIME_R option is true, then SQLite uses the threadsafe
  localtime_r() library routine instead of localtime()
  to help implement the localtime modifier
  to the built-in date and time functions.

HAVE_LOCALTIME_S
  If the HAVE_LOCALTIME_S option is true, then SQLite uses the threadsafe
  localtime_s() library routine instead of localtime()
  to help implement the localtime modifier
  to the built-in date and time functions.

HAVE_MALLOC_USABLE_SIZE
  If the HAVE_MALLOC_USABLE_SIZE option is true, then SQLite tries uses the
  malloc_usable_size() interface to find the size of a memory allocation obtained
  from the standard-library malloc() or realloc() routines.  This option is only
  applicable if the standard-library malloc() is used.  On Apple systems,
  "zone malloc" is used instead, and so this option is not applicable.  And, of
  course, if the application supplies its own malloc implementation using
  SQLITE_CONFIG_MALLOC then this option has no effect.
  
  If the HAVE_MALLOC_USABLE_SIZE option is omitted or is false, then SQLite
  uses a wrapper around system malloc() and realloc() that enlarges each allocation
  by 8 bytes and writes the size of the allocation in the initial 8 bytes, and
  then SQLite also implements its own home-grown version of malloc_usable_size()
  that consults that 8-byte prefix to find the allocation size.  This approach
  works but it is suboptimal.  Applications are encouraged to use
  HAVE_MALLOC_USABLE_SIZE whenever possible.

HAVE_STRCHRNUL
  If the HAVE_STRCHRNUL option is true, then SQLite uses the strchrnul() library
  function.  If this option is missing or false, then SQLite substitutes its own
  home-grown implementation of strchrnul().

HAVE_UTIME
  If the HAVE_UTIME option is true, then the built-in but non-standard
  "unix-dotfile" VFS will use the utime() system call, instead of utimes(),
  to set the last access time on the lock file.

SQLITE_BYTEORDER=(0|1234|4321)
  SQLite needs to know if the native byte order of the target CPU is
  big-endian or little-ending.  The SQLITE_BYTEORDER preprocessor is set
  to 4321 for big-endian machines and 1234 for little-endian machines, or
  it can be 0 to mean that the byte order must be determined at run-time.
  There are #ifdefs in the code that set SQLITE_BYTEORDER automatically
  for all common platforms and compilers.  However, it may be advantageous
  to set SQLITE_BYTEORDER appropriately when compiling SQLite for obscure
  targets.  If the target byte order cannot be determined at compile-time,
  then SQLite falls back to doing run-time checks, which always work, though
  with a small performance penalty.


c2: Compile-time Options
c3: 4. Options To Set Default Parameter Values
c5: compile.html#_options_to_set_default_parameter_values
c4:
SQLITE_DEFAULT_AUTOMATIC_INDEX=&lt;0 or 1&gt;
  This macro determines the initial setting for PRAGMA automatic_index
  for newly opened database connections.
  For all versions of SQLite through 3.7.17,
  automatic indices are normally enabled for new database connections if
  this compile-time option is omitted.
  However, that might change in future releases of SQLite.
  See also: SQLITE_OMIT_AUTOMATIC_INDEX

SQLITE_DEFAULT_AUTOVACUUM=&lt;0 or 1 or 2&gt;
  This macro determines if SQLite creates databases with the
  auto_vacuum flag set by default to OFF (0), FULL (1), or
  INCREMENTAL (2). The default value is 0 meaning that databases
  are created with auto-vacuum turned off.
  In any case the compile-time default may be overridden by the
  PRAGMA auto_vacuum command.

SQLITE_DEFAULT_CACHE_SIZE=&lt;N&gt;
  This macro sets the default maximum size of the page-cache for each attached
  database.  A positive value means that the limit is N page.  If N is negative
  that means to limit the cache size to -N*1024 bytes.
  The suggested maximum cache size can be overridden by the
  PRAGMA cache_size command. The default value is -2000, which translates
  into a maximum of 2048000 bytes per cache.

SQLITE_DEFAULT_FILE_FORMAT=&lt;1 or 4&gt;
  The default schema format number used by SQLite when creating
  new database files is set by this macro.  The schema formats are all
  very similar.  The difference between formats 1 and 4 is that format
  4 understands descending indices and has a tighter encoding for
  boolean values.

  All versions of SQLite since 3.3.0 (2006-01-10)
  can read and write any schema format
  between 1 and 4.  But older versions of SQLite might not be able to
  read formats greater than 1.  So that older versions of SQLite will
  be able to read and write database files created by newer versions
  of SQLite, the default schema format was set to 1 for SQLite versions
  through 3.7.9 (2011-11-01).  Beginning with
  version 3.7.10 (2012-01-16), the default
  schema format is 4.

  The schema format number for a new database can be set at runtime using
  the PRAGMA legacy_file_format command.

SQLITE_DEFAULT_FILE_PERMISSIONS=N
  The default numeric file permissions for newly created database files
  under unix.  If not specified, the default is 0644 which means that
  the files is globally readable but only writable by the creator.

SQLITE_DEFAULT_FOREIGN_KEYS=&lt;0 or 1&gt;
  This macro determines whether enforcement of
  foreign key constraints is enabled or disabled by default for
  new database connections.  Each database connection can always turn
  enforcement of foreign key constraints on and off and run-time using
  the foreign_keys pragma.  Enforcement of foreign key constraints
  is normally off by default, but if this compile-time parameter is
  set to 1, enforcement of foreign key constraints will be on by default.

SQLITE_DEFAULT_MMAP_SIZE=N
  This macro sets the default limit on the amount of memory that
  will be used for memory-mapped I/O
  for each open database file.  If the N
  is zero, then memory mapped I/O is disabled by default.  This
  compile-time limit and the SQLITE_MAX_MMAP_SIZE can be modified
  at start-time using the
  sqlite3_config(SQLITE_CONFIG_MMAP_SIZE) call, or at run-time
  using the mmap_size pragma.

SQLITE_DEFAULT_JOURNAL_SIZE_LIMIT=&lt;bytes&gt;
  This option sets the size limit on rollback journal files in
  persistent journal mode and
  exclusive locking mode and on the size of the
  write-ahead log file in WAL mode. When this
  compile-time option is omitted there is no upper bound on the
  size of the rollback journals or write-ahead logs.
  The journal file size limit
  can be changed at run-time using the journal_size_limit pragma.

SQLITE_DEFAULT_LOCKING_MODE=&lt;1 or 0&gt;
  If set to 1, then the default locking_mode is set to EXCLUSIVE.
  If omitted or set to 0 then the default locking_mode is NORMAL.

SQLITE_DEFAULT_LOOKASIDE=SZ,N
  Sets the default size of the lookaside memory allocator memory pool
  to N entries of SZ bytes each.  This setting can be modified at
  start-time using sqlite3_config(SQLITE_CONFIG_LOOKASIDE) and/or
  as each database connection is opened using
  sqlite3_db_config(db, SQLITE_DBCONFIG_LOOKASIDE).

SQLITE_DEFAULT_MEMSTATUS=&lt;1 or 0&gt;
  This macro is used to determine whether or not the features enabled and
  disabled using the SQLITE_CONFIG_MEMSTATUS argument to sqlite3_config()
  are available by default. The default value is 1 (SQLITE_CONFIG_MEMSTATUS
  related features enabled).
  
  The sqlite3_memory_used() and sqlite3_memory_highwater() interfaces,
  the sqlite3_status64(SQLITE_STATUS_MEMORY_USED) interface,
  and the SQLITE_MAX_MEMORY compile-time option are all non-functional
  when memory usage tracking is disabled.

SQLITE_DEFAULT_PCACHE_INITSZ=N
  This macro determines the number of pages initially allocated by the
  page cache module when SQLITE_CONFIG_PAGECACHE configuration option is
  not use and memory for the page cache is obtained from sqlite3_malloc()
  instead.  The number of pages set by this macro are allocated in a single
  allocation, which reduces the load on the memory allocator.

SQLITE_DEFAULT_PAGE_SIZE=&lt;bytes&gt;
  This macro is used to set the default page-size used when a
  database is created. The value assigned must be a power of 2. The
  default value is 4096. The compile-time default may be overridden at
  runtime by the PRAGMA page_size command.

SQLITE_DEFAULT_SYNCHRONOUS=&lt;0-3&gt;
  This macro determines the default value of the
  PRAGMA synchronous setting.  If not overridden at compile-time,
  the default setting is 2 (FULL).

SQLITE_DEFAULT_WAL_SYNCHRONOUS=&lt;0-3&gt;
  This macro determines the default value of the
  PRAGMA synchronous setting for database files that open in
  WAL mode.  If not overridden at compile-time, this value is the
  same as SQLITE_DEFAULT_SYNCHRONOUS.
  
  If SQLITE_DEFAULT_WAL_SYNCHRONOUS differs from SQLITE_DEFAULT_SYNCHRONOUS,
  and if the application has not modified the synchronous setting for
  the database file using the PRAGMA synchronous statement, then
  the synchronous setting is changed to value defined by
  SQLITE_DEFAULT_WAL_SYNCHRONOUS when the database connection switches
  into WAL mode for the first time.
  If the SQLITE_DEFAULT_WAL_SYNCHRONOUS value is not overridden at
  compile-time, then it will always be the same as
  SQLITE_DEFAULT_SYNCHRONOUS and so no automatic synchronous setting
  changes will ever occur.

SQLITE_DEFAULT_WAL_AUTOCHECKPOINT=&lt;pages&gt;
  This macro sets the default page count for the WAL
  automatic checkpointing feature.  If unspecified,
  the default page count is 1000.

SQLITE_DEFAULT_WORKER_THREADS=N
  This macro sets the default value for
  the SQLITE_LIMIT_WORKER_THREADS parameter.  The SQLITE_LIMIT_WORKER_THREADS
  parameter sets the maximum number of auxiliary threads that a single
  prepared statement will launch to assist it with a query.  If not specified,
  the default maximum is 0.
  The value set here cannot be more than SQLITE_MAX_WORKER_THREADS.

SQLITE_DQS=N
  This macro determines the default values for
  SQLITE_DBCONFIG_DQS_DDL and SQLITE_DBCONFIG_DQS_DML, which
  in turn determine how SQLite handles each double-quoted string literal.
  The "DQS" name stands for
  "Double-Quoted String".
  The N argument should be an integer 0, 1, 2, or 3.
  
  SQLITE_DQSDouble-Quoted Strings Allowed
      Remarks
  In DDLIn DML
  3yesyesdefault
  2yesno&nbsp;
  1noyes&nbsp;
  0nonorecommended
  
  The recommended setting is 0, meaning that double-quoted
  strings are disallowed in all contexts.  However, the default
  setting is 3 for maximum compatibility with legacy applications.

SQLITE_EXTRA_DURABLE
  The SQLITE_EXTRA_DURABLE compile-time option that used to cause the default
  PRAGMA synchronous setting to be EXTRA, rather than FULL.  This option
  is no longer supported.  Use
  SQLITE_DEFAULT_SYNCHRONOUS=3 instead.

SQLITE_FTS3_MAX_EXPR_DEPTH=N
  This macro sets the maximum depth of the search tree that corresponds to
  the right-hand side of the MATCH operator in an FTS3 or FTS4 full-text
  index.  The full-text search uses a recursive algorithm, so the depth of
  the tree is limited to prevent using too much stack space.  The default
  limit is 12.  This limit is sufficient for up to 4095 search terms on the
  right-hand side of the MATCH operator and it holds stack space usage to
  less than 2000 bytes.
  
  For ordinary FTS3/FTS4 queries, the search tree depth is approximately
  the base-2 logarithm of the number of terms in the right-hand side of the
  MATCH operator.  However, for phrase queries and NEAR queries the
  search tree depth is linear in the number of right-hand side terms.
  So the default depth limit of 12 is sufficient for up to 4095 ordinary
  terms on a MATCH, it is only sufficient for 11 or 12 phrase or NEAR
  terms.  Even so, the default is more than enough for most application.

SQLITE_LIKE_DOESNT_MATCH_BLOBS
  This compile-time option causes the LIKE operator to always return
  False if either operand is a BLOB.  The default behavior of LIKE
  is that BLOB operands are cast to TEXT before the comparison is done.
  
  This compile-time option makes SQLite run more efficiently when processing
  queries that use the LIKE operator, at the expense of breaking backwards
  compatibility.  However, the backwards compatibility break may be only
  a technicality.  There was a long-standing bug in the LIKE processing logic
  (see https://www.sqlite.org/src/info/05f43be8fdda9f) that caused it to
  misbehavior for BLOB operands and nobody observed that bug in nearly
  10 years of active use.  So for more users, it is probably safe to
  enable this compile-time option and thereby save a little CPU time
  on LIKE queries.
  
  This compile-time option affects the SQL LIKE operator only and has
  no impact on the sqlite3_strlike() C-language interface.

SQLITE_MAX_MEMORY=N
  This option limits the total amount of memory that SQLite will request
  from malloc() to N bytes.  Any attempt by SQLite to allocate
  new memory that would cause the sum of all allocations held by SQLite to exceed
  N bytes will result in an out-of-memory error.
  This is a hard upper limit.  See also the sqlite3_soft_heap_limit()
  interface.
  
  This limit is only functional if memory usage statistics are available via
  the sqlite3_memory_used() and sqlite3_status64(SQLITE_STATUS_MEMORY_USED)
  interfaces.  Without that memory usage information, SQLite has no way of
  knowing when it is about to go over the limit, and thus is unable to prevent
  the excess memory allocation.  Memory usage tracking is turned on by default,
  but can be disabled at compile-time using the SQLITE_DEFAULT_MEMSTATUS option,
  or at start-time using sqlite3_config(SQLITE_CONFIG_MEMSTATUS).

SQLITE_MAX_MMAP_SIZE=N
  This macro sets a hard upper bound on the amount of address space that
  can be used by any single database for memory-mapped I/O.
  Setting this value to 0 completely disables memory-mapped I/O and
  causes logic associated with memory-mapped I/O to be omitted from the
  build.  This option does change the default memory-mapped I/O address
  space size (set by SQLITE_DEFAULT_MMAP_SIZE or
  sqlite3_config(SQLITE_CONFIG_MMAP_SIZE) or the
  run-time memory-mapped I/O address space size (set by
  sqlite3_file_control(SQLITE_FCNTL_MMAP_SIZE) or
  PRAGMA mmap_size) as long as those other settings are less than the
  maximum value defined here.

SQLITE_MAX_SCHEMA_RETRY=N
  Whenever the database schema changes, prepared statements are automatically
  reprepared to accommodate the new schema.  There is a race condition here
  in that if one thread is constantly changing the schema, another thread
  might spin on reparses and repreparations of a prepared statement and
  never get any real work done.  This parameter prevents an infinite loop
  by forcing the spinning thread to give up after a fixed number of attempts
  at recompiling the prepared statement.  The default setting is 50 which is
  more than adequate for most applications.

SQLITE_MAX_WORKER_THREADS=N
  Set an upper bound on the sqlite3_limit(db,SQLITE_LIMIT_WORKER_THREADS,N)
  setting that determines the maximum number of auxiliary threads that a single
  prepared statement will use to aid with CPU-intensive computations
  (mostly sorting).  See also the SQLITE_DEFAULT_WORKER_THREADS options.

SQLITE_MEMDB_DEFAULT_MAXSIZE=N
  Set the default size limit (in bytes) for in-memory databases created using
  sqlite3_deserialize().  This is just the default.  The limit can be
  changed at start-time using
  sqlite3_config(SQLITE_CONFIG_MEMDB_MAXSIZE,N)
  or at run-time for individual databases using the
  SQLITE_FCNTL_SIZE_LIMIT file-control.
  If no default is specified, 1073741824 is used.

SQLITE_MINIMUM_FILE_DESCRIPTOR=N
  The unix VFS will never use a file descriptor less than N.  The
  default value of N is 3.
  
  Avoiding the use of low-numbered file descriptors is a defense against
  accidental database corruption.  If a database file was opened using
  file descriptor 2, for example, and then an assert() failed and invoked
  write(2,...), that would likely cause database corruption by overwriting
  part of the database file with the assertion error message.  Using only
  higher-valued file descriptors avoids this potential problem.  The
  protection against
  using low-numbered file descriptors can be disabled by setting this
  compile-time option to 0.

SQLITE_POWERSAFE_OVERWRITE=&lt;0 or 1&gt;
  This option changes the default assumption about powersafe overwrite
  for the underlying filesystems for the unix and windows VFSes.
  Setting SQLITE_POWERSAFE_OVERWRITE to 1 causes SQLite to assume that
  application-level writes cannot changes bytes outside the range of
  bytes written even if the write occurs just before a power loss.
  With SQLITE_POWERSAFE_OVERWRITE set to 0, SQLite assumes that other
  bytes in the same sector with a written byte might be changed or
  damaged by a power loss.

SQLITE_PRINTF_PRECISION_LIMIT=N
  This option limits the maximum width and precision of substitutions
  for the printf() SQL function and the other C-language string
  formatting functions such as sqlite3_mprintf() and
  sqlite3_str_appendf().  This is turn can prevent a hostile or
  malfunctioning script from using excessive memory by invoking
  a format such as: "printf('%*s',2147483647,'hi')".
  A value for N of around 100000 is normally sufficient.
  
  The printf() SQL function is subject to the SQLITE_LIMIT_LENGTH
  limit of sqlite3_limit().  Hence any printf() result with a
  width or precision more than the SQLITE_LIMIT_LENGTH will cause
  an SQLITE_TOOBIG error.  However, the low-level formatting
  for the printf() function is done by a subroutine that does not
  have access to SQLITE_LIMIT_LENGTH.  So the low-level formatting
  is done into a memory allocation that might be considerably larger
  than SQLITE_LIMIT_LENGTH and the SQLITE_LIMIT_LENGTH check is only
  performed after all formatting is complete.  Thus there might be a
  transient buffer that exceeds SQLITE_LIMIT_LENGTH.  The
  SQLITE_PRINTF_PRECISION_LIMIT option is an additional check
  that prevents excess sizes for the transient buffer used inside
  the low-level formatting subroutine, prior to the
  SQLITE_LIMIT_LENGTH check.
  
  Be careful not to set SQLITE_PRINTF_PRECISION_LIMIT too low.
  SQLite uses its built-in printf() functionality to format the text
  of CREATE statements stored in the sqlite_schema table.  So
  SQLITE_PRINTF_PRECISION_LIMIT should be at least as big as the largest
  table, index, view, or trigger definition that you are likely to
  encounter.
  
  No error is raised if a width or precision exceeds
  SQLITE_PRINTF_PRECISION_LIMIT.  Instead, the large width or
  precision is silently truncated.
  
  The default value for SQLITE_PRINTF_PRECISION_LIMIT is 2147483647
  (0x7fffffff).


SQLITE_QUERY_PLANNER_LIMIT=N
  As part of the query planning process, SQLite enumerates all usable
  combinations of indexes and WHERE-clause constraints.  For certain
  pathological queries, the number of these index-and-constraint combinations
  can be very large, resulting in slow performance by the query planner.
  The SQLITE_QUERY_PLANNER_LIMIT value (in conjunction with the
  related SQLITE_QUERY_PLANNER_LIMIT_INCR setting) limits the number
  of index-and-constraint combinations that the query planner will
  consider, in order to prevent the query planner from using excess
  CPU time.  The default value for SQLITE_QUERY_PLANNER_LIMIT is set
  high enough so that is never reached for real-world queries.  The
  query planner search limit only applies to queries that are deliberately
  crafted to use excess planning time.

SQLITE_QUERY_PLANNER_LIMIT_INCR=N
  The SQLITE_QUERY_PLANNER_LIMIT option sets an initial baseline value
  for the maximum number of index-and-constraint combinations that the
  query planner consider.  The baseline query planner limit is increased
  by SQLITE_QUERY_PLANNER_LIMIT_INCR prior to processing each table of a
  join so that each table is guaranteed to be able to propose at least
  some index-and-constraint combinations to the optimizer even if prior
  tables of the join have exhausted the baseline limit.  The default
  value for both this compile-time option and the
  SQLITE_QUERY_PLANNER_LIMIT option are set high enough so that they should
  never be reached for real-world queries.

SQLITE_REVERSE_UNORDERED_SELECTS
  This option causes the PRAGMA reverse_unordered_selects setting to be
  enabled by default.  When enabled, SELECT statements that lack an
  ORDER BY clause will run in reverse order.
  This option is useful for detecting when applications (incorrectly)
  assume that the order of rows in a SELECT without an ORDER BY clause
  will always be the same.

SQLITE_SORTER_PMASZ=N
  If multi-threaded processing is enabled via the
  PRAGMA threads setting, then sort operations will
  attempt to start helper threads when the amount of content
  to be sorted exceeds the minimum of the cache_size and PMA Size
  determined by the SQLITE_CONFIG_PMASZ start-time option.
  This compile-time option sets the default value for the
  SQLITE_CONFIG_PMASZ start-time option.
  The default value is 250.

SQLITE_STMTJRNL_SPILL=N
  The SQLITE_STMTJRNL_SPILL compile-time option determines the
  default setting of the SQLITE_CONFIG_STMTJRNL_SPILL start-time
  setting.  That setting determines the size threshold above which
  statement journals are moved from memory to disk.

SQLITE_WIN32_MALLOC
  This option enables the use of the Windows Heap API functions for memory
  allocation instead of the standard library malloc() and free() routines.

YYSTACKDEPTH=&lt;max_depth&gt;
  This macro sets the maximum depth of the LALR(1) stack used by
  the SQL parser within SQLite.  The default value is 100.  A typical
  application will use less than about 20 levels of the stack.
  Developers whose applications contain SQL statements that
  need more than 100 LALR(1) stack entries should seriously
  consider refactoring their SQL as it is likely to be well beyond
  the ability of any human to comprehend.


c2: Compile-time Options
c3: 5. Options To Set Size Limits
c5: compile.html#_options_to_set_size_limits
c4:
There are compile-time options that will set upper bounds
on the sizes of various structures in SQLite.  The compile-time
options normally set a hard upper bound that can be changed
at run-time on individual database connections using the
sqlite3_limit() interface.

The compile-time options for setting upper bounds are
documented separately.  The following is a list of
the available settings:


 SQLITE_MAX_ATTACHED  
 SQLITE_MAX_COLUMN  
 SQLITE_MAX_COMPOUND_SELECT  
 SQLITE_MAX_EXPR_DEPTH  
 SQLITE_MAX_FUNCTION_ARG  
 SQLITE_MAX_LENGTH  
 SQLITE_MAX_LIKE_PATTERN_LENGTH  
 SQLITE_MAX_PAGE_COUNT  
 SQLITE_MAX_SQL_LENGTH  
 SQLITE_MAX_VARIABLE_NUMBER


c2: Compile-time Options
c3: 6. Options To Control Operating Characteristics
c5: compile.html#_options_to_control_operating_characteristics
c4:
SQLITE_4_BYTE_ALIGNED_MALLOC
  On most systems, the malloc() system call returns a buffer that is
  aligned to an 8-byte boundary.  But on some systems (ex: windows) malloc()
  returns 4-byte aligned pointer.  This compile-time option must be used
  on systems that return 4-byte aligned pointers from malloc().

SQLITE_CASE_SENSITIVE_LIKE
  If this option is present, then the built-in LIKE operator will be
  case sensitive.  This same effect can be achieved at run-time using
  the case_sensitive_like pragma.

SQLITE_DIRECT_OVERFLOW_READ
  When this option is present, content contained in
  overflow pages of the database file is read directly from disk,
  bypassing the page cache, during read transactions.  In applications
  that do a lot of reads of large BLOBs, this option might improve read
  performance.

SQLITE_HAVE_ISNAN
  If this option is present, then SQLite will use the isnan() function from
  the system math library.  This is an alias for the HAVE_ISNAN configuration
  option.

SQLITE_MAX_ALLOCATION_SIZE=N
  This compile-time option sets an upper bound on the size of memory
  allocations that can be requested using sqlite3_malloc64(),
  sqlite3_realloc64(), and similar.  The default value is
  2,147,483,391 (0x7ffffeff) and this should be considered an
  upper bound.  Most applications can get by with a maximum allocation
  size of a few million bytes.
  
  This is a limit on the maximum size of any single memory allocation.
  It is not a limit on the total amount of memory allocated.
  
  Reducing the maximum size of individual memory allocations provides
  extra defense against denial-of-service attacks that attempt to exhaust
  system memory by doing many large allocations.  It is also an extra layer
  of defense against application bugs where the size of a memory allocation
  is computed using a signed 32-bit integer that could overflow &rarr;
  with a small maximum allocation size, such buggy memory allocation size
  computations are likely to be spotted sooner due to out-of-memory errors
  and before the integer actually overflows.

SQLITE_OS_OTHER=&lt;0 or 1&gt;
  The option causes SQLite to omit its built-in operating system interfaces
  for Unix, Windows, and OS/2.  The resulting library will have no default
  operating system interface.  Applications must use
  sqlite3_vfs_register() to register an appropriate interface before
  using SQLite.  Applications must also supply implementations for the
  sqlite3_os_init() and sqlite3_os_end() interfaces.  The usual practice
  is for the supplied sqlite3_os_init() to invoke sqlite3_vfs_register().
  SQLite will automatically invoke sqlite3_os_init() when it initializes.

  This option is typically used when building SQLite for an embedded
  platform with a custom operating system.

SQLITE_SECURE_DELETE
  This compile-time option changes the default setting of the
  secure_delete pragma.  When this option is not used, secure_delete defaults
  to off.  When this option is present, secure_delete defaults to on.

  The secure_delete setting causes deleted content to be overwritten with
  zeros.  There is a small performance penalty since additional I/O
  must occur.  On the other hand, secure_delete can prevent fragments of
  sensitive information from lingering in unused parts of the database file
  after it has been deleted.  See the documentation on the
  secure_delete pragma for additional information.

SQLITE_THREADSAFE=&lt;0 or 1 or 2&gt;
  This option controls whether or not code is included in SQLite to
  enable it to operate safely in a multithreaded environment.  The
  default is SQLITE_THREADSAFE=1 which is safe for use in a multithreaded
  environment.  When compiled with SQLITE_THREADSAFE=0 all mutexing code
  is omitted and it is unsafe to use SQLite in a multithreaded program.
  When compiled with SQLITE_THREADSAFE=2, SQLite can be used in a multithreaded
  program so long as no two threads attempt to use the same
  database connection (or any prepared statements derived from
  that database connection) at the same time.

  To put it another way, SQLITE_THREADSAFE=1 sets the default
  threading mode to Serialized.  SQLITE_THREADSAFE=2 sets the default
  threading mode to Multi-threaded.  And SQLITE_THREADSAFE=0 sets the
  threading mode to Single-threaded.

  The value of SQLITE_THREADSAFE can be determined at run-time
  using the sqlite3_threadsafe() interface.

  When SQLite has been compiled with SQLITE_THREADSAFE=1 or
  SQLITE_THREADSAFE=2 then the threading mode
  can be altered at run-time using the sqlite3_config() interface together
  with one of these verbs:

  
  SQLITE_CONFIG_SINGLETHREAD
  SQLITE_CONFIG_MULTITHREAD
  SQLITE_CONFIG_SERIALIZED
  

  The SQLITE_OPEN_NOMUTEX and
  SQLITE_OPEN_FULLMUTEX flags to sqlite3_open_v2() can also be used
  to adjust the threading mode of individual database connections
  at run-time.

  Note that when SQLite is compiled with SQLITE_THREADSAFE=0, the code
  to make SQLite threadsafe is omitted from the build.  When this occurs,
  it is impossible to change the threading mode at start-time or run-time.

  See the threading mode documentation for additional information
  on aspects of using SQLite in a multithreaded environment.

SQLITE_TEMP_STORE=&lt;0 through 3&gt;
  This option controls whether temporary files are stored on disk or
  in memory.  The meanings for various settings of this compile-time
  option are as follows:

  
  SQLITE_TEMP_STOREMeaning
  0Always use temporary files
  1Use files by default but allow the
  PRAGMA temp_store command to override
  2Use memory by default but allow the
  PRAGMA temp_store command to override
  3Always use memory
  

  The default setting is 1.
  Additional information can be found in tempfiles.html.

SQLITE_TRACE_SIZE_LIMIT=N
  If this macro is defined to a positive integer N, then the length of
  strings and BLOB that are expanded into parameters in the output of
  sqlite3_trace() is limited to N bytes.

SQLITE_TRUSTED_SCHEMA=&lt;0 or 1&gt;
  This macro determines the default value for the
  SQLITE_DBCONFIG_TRUSTED_SCHEMA and PRAGMA trusted_schema setting.
  If no alternative is specified, the trusted-schema setting defaults
  to ON (a value of 1) for legacy compatibility.  However, for best
  security, systems that implement
  application-defined SQL functions and/or virtual tables should
  consider changing the default to OFF.

SQLITE_USE_URI
  This option causes the URI filename process logic to be enabled by
  default.


c2: Compile-time Options
c3: 7. Options To Enable Features Normally Turned Off
c5: compile.html#_options_to_enable_features_normally_turned_off
c4:
SQLITE_ALLOW_URI_AUTHORITY
  URI filenames normally throws an error if the authority section is
  not either empty or "localhost".  However, if SQLite is compiled with
  the SQLITE_ALLOW_URI_AUTHORITY compile-time option, then the URI is
  converted into a Uniform Naming Convention (UNC) filename and passed
  down to the underlying operating system that way.
  
  Some future versions of SQLite may change to enable this feature
  by default.

SQLITE_ALLOW_COVERING_INDEX_SCAN=&lt;0 or 1&gt;
  This C-preprocess macro determines the default setting of the
  SQLITE_CONFIG_COVERING_INDEX_SCAN configuration setting.  It defaults
  to 1 (on) which means that covering indices are used for full table
  scans where possible, in order to reduce I/O and improve performance.
  However, the use of a covering index for a full scan will cause results
  to appear in a different order from legacy, which could cause some
  (incorrectly-coded) legacy applications to break.  Hence, the covering
  index scan option can be disabled at compile-time on systems that what
  to minimize their risk of exposing errors in legacy applications.

SQLITE_ENABLE_8_3_NAMES=&lt;1 or 2&gt;
  If this C-preprocessor macro is defined, then extra code is
  included that allows SQLite to function on a filesystem that
  only support 8+3 filenames.  If the value of this macro is 1,
  then the default behavior is to continue to use long filenames and
  to only use 8+3 filenames if the
  database connection is opened using URI filenames with
  the "8_3_names=1" query parameter.  If the value of
  this macro is 2, then the use of 8+3 filenames becomes the default
  but may be disabled on using the 8_3_names=0 query parameter.

SQLITE_ENABLE_API_ARMOR
  When defined, this C-preprocessor macro activates extra code that
  attempts to detect misuse of the SQLite API, such as passing in NULL
  pointers to required parameters or using objects after they have been
  destroyed.  When this option is enabled and an illegal API usage
  is detected, the interface will typically return SQLITE_MISUSE.
  
  The SQLITE_ENABLE_API_ARMOR option does not guarantee that all 
  illegal API usages will be detected.  Even when
  SQLITE_ENABLE_API_ARMOR is enabled, passing incorrect values
  into the C-language APIs can cause a process crash due to segmentation
  fault or null-pointer deference or other reasons.  The
  SQLITE_ENABLE_API_ARMOR compile-time option is intended as an aid
  for application testing and debugging option.  Applications
  should not depend SQLITE_ENABLE_API_ARMOR for safety.
  SQLITE_ENABLE_API_ARMORE is appropriate as a second line of
  defense against application bugs, but it should not be the only
  defense.  If any SQLite interface returns SQLITE_MISUSE, that
  indicates that the application is using SQLite contrary to
  the spec and that the application contains a bug.  The SQLITE_MISUSE
  return provides the application with the opportunity to respond
  gracefully to that bug, rather than simply crashing the process,
  but nothing more.  Applications should not make use of nor depend
  upon SQLITE_MISUSE for routine processing.

SQLITE_ENABLE_ATOMIC_WRITE
  If this C-preprocessor macro is defined and if the
  xDeviceCharacteristics method of sqlite3_io_methods object for
  a database file reports (via one of the SQLITE_IOCAP_ATOMIC bits)
  that the filesystem supports atomic writes and if a transaction
  involves a change to only a single page of the database file,
  then the transaction commits with just a single write request of
  a single page of the database and no rollback journal is created
  or written.  On filesystems that support atomic writes, this
  optimization can result in significant speed improvements for
  small updates.  However, few filesystems support this capability
  and the code paths that check for this capability slow down write
  performance on systems that lack atomic write capability, so this
  feature is disabled by default.

SQLITE_ENABLE_BATCH_ATOMIC_WRITE
  This compile-time option enables SQLite to take advantage batch
  atomic write capabilities in the underlying filesystem.  As of
  SQLite version 3.21.0 (2017-10-24) this is only supported on
  F2FS.  However, the interface
  is implemented generically, using sqlite3_file_control() with
  SQLITE_FCNTL_BEGIN_ATOMIC_WRITE and SQLITE_FCNTL_COMMIT_ATOMIC_WRITE
  so the capability can be added to other filesystem times in the
  future.  When this option is enabled, SQLite automatically detects
  that the underlying filesystem supports batch atomic writes, and
  when it does so it avoids writing the rollback journal for transaction
  control.  This can make transactions over twice as fast, while
  simultaneously reducing wear on SSD storage devices.

  Future versions of SQLite might enable the batch-atomic-write
  capability by default, at which point this compile-time option
  will become superfluous.

SQLITE_ENABLE_BYTECODE_VTAB
  This option enables the bytecode and tables_used virtual tables.

SQLITE_ENABLE_COLUMN_METADATA
  When this C-preprocessor macro is defined, SQLite includes some
  additional APIs that provide convenient access to meta-data about
  tables and queries.  The APIs that are enabled by this option are:

  
   sqlite3_column_database_name() 
   sqlite3_column_database_name16() 
   sqlite3_column_table_name() 
   sqlite3_column_table_name16() 
   sqlite3_column_origin_name() 
   sqlite3_column_origin_name16() 
  

SQLITE_ENABLE_DBPAGE_VTAB
  This option enables the SQLITE_DBPAGE virtual table.

SQLITE_ENABLE_DBSTAT_VTAB
  This option enables the dbstat virtual table.

SQLITE_ENABLE_DESERIALIZE
  This option was formerly used to enable
  the sqlite3_serialize() and sqlite3_deserialize()
  interfaces.  However, as of SQLite 3.36.0 (2021-06-18)
  those interfaces are enabled by default and a new
  compile-time option SQLITE_OMIT_DESERIALIZE is added
  to omit them.

SQLITE_ENABLE_EXPLAIN_COMMENTS
  This option adds extra logic to SQLite that inserts comment text into the
  output of EXPLAIN.  These extra comments use extra memory, thus
  making prepared statements larger and very slightly slower, and so they are
  turned off by default and in most application.  But some applications, such
  as the command-line shell for SQLite, value clarity of EXPLAIN output
  over raw performance and so this compile-time option is available to them.
  The SQLITE_ENABLE_EXPLAIN_COMMENTS compile-time option is also enabled
  automatically if SQLITE_DEBUG is enabled.

SQLITE_ENABLE_FTS3
  When this option is defined in the amalgamation, versions 3 and 4
  of the full-text search engine are added to the build automatically.

SQLITE_ENABLE_FTS3_PARENTHESIS
  This option modifies the query pattern parser in FTS3 such that it
  supports operators AND and NOT (in addition to the usual OR and NEAR)
  and also allows query expressions to contain nested parenthesis.

SQLITE_ENABLE_FTS3_TOKENIZER
  This option enables the two-argument version of the fts3_tokenizer()
  interface.  The second argument to fts3_tokenizer() is suppose to be a
  pointer to a function (encoded as a BLOB) that implements an
  application defined tokenizer.  If hostile actors are able to run
  the two-argument version of fts3_tokenizer() with an arbitrary second
  argument, they could use crash or take control of the process.
  
  Because of security concerns, the two-argument fts3_tokenizer() feature
  was disabled beginning with Version 3.11.0 (2016-02-15)
  unless this compile-time option is used.
  Version 3.12.0 (2016-03-29) added the
  sqlite3_db_config(db,SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER,1,0) interface
  that activates the two-argument version of fts3_tokenizer()
  for a specific database connection at run-time.

SQLITE_ENABLE_FTS4
  When this option is defined in the amalgamation, versions 3 and 4
  of the full-text search engine are added to the build automatically.

SQLITE_ENABLE_FTS5
  When this option is defined in the amalgamation, versions 5
  of the full-text search engine (fts5) is added to the build automatically.

SQLITE_ENABLE_GEOPOLY
  When this option is defined in the amalgamation, the Geopoly extension
  is included in the build.

SQLITE_ENABLE_ICU
  This option causes the
  International Components for Unicode
  or "ICU" extension to SQLite to be added to the build.

SQLITE_ENABLE_IOTRACE
  When both the SQLite core and the Command Line Interface (CLI) are both
  compiled with this option, then the CLI provides an extra command
  named ".iotrace" that provides a low-level log of I/O activity.
  This option is experimental and may be discontinued in a future release.

SQLITE_ENABLE_MATH_FUNCTIONS
  This macro enables the built-in SQL math functions.  This option
  is automatically added to the Makefile by the configure script on unix platforms,
  unless the --disable-math option is used.
  This option is also included on Windows builds using the
  "Makefile.msc" makefile for nmake.

SQLITE_ENABLE_JSON1
  This compile-time option is a no-op.  Prior to SQLite version 3.38.0
  (2022-02-22), it was necessary to compile with this option in order
  to include the JSON SQL functions in the build.  However, beginning
  with SQLite version 3.38.0, those functions are included by default.
  Use the -DSQLITE_OMIT_JSON option to omit them.

SQLITE_ENABLE_LOCKING_STYLE
  This option enables additional logic in the OS interface layer for
  Mac OS X. The additional logic attempts to determine the type of the
  underlying filesystem and choose and alternative locking strategy
  that works correctly for that filesystem type. Five locking strategies
  are available:

  
     POSIX locking style. This is the default locking style and the
         style used by other (non Mac OS X) Unixes. Locks are obtained and
         released using the fcntl() system call.

     AFP locking style. This locking style is used for network file
         systems that use the AFP (Apple Filing Protocol) protocol. Locks
         are obtained by calling the library function _AFPFSSetLock().

     Flock locking style. This is used for file-systems that do not
         support POSIX locking style. Locks are obtained and released using
         the flock() system call.

     Dot-file locking style. This locking style is used when neither
         flock nor POSIX locking styles are supported by the file system.
         Database locks are obtained by creating and entry in the file-system
         at a well-known location relative to the database file (a "dot-file")
         and relinquished by deleting the same file.

     No locking style. If none of the above can be supported, this
         locking style is used. No database locking mechanism is used. When
         this system is used it is not safe for a single database to be
         accessed by multiple clients.
  

  Additionally, five extra VFS implementations are provided as well as the
  default. By specifying one of the extra VFS implementations
  when calling sqlite3_open_v2(), an application may bypass the file-system
  detection logic and explicitly select one of the above locking styles. The
  five extra VFS implementations are called "unix-posix", "unix-afp",
  "unix-flock", "unix-dotfile" and "unix-none".

SQLITE_ENABLE_MEMORY_MANAGEMENT
  This option adds extra logic to SQLite that allows it to release unused
  memory upon request.  This option must be enabled in order for the
  sqlite3_release_memory() interface to work.  If this compile-time
  option is not used, the sqlite3_release_memory() interface is a
  no-op.

SQLITE_ENABLE_MEMSYS3
  This option includes code in SQLite that implements an alternative
  memory allocator.  This alternative memory allocator is only engaged
  when the SQLITE_CONFIG_HEAP option to sqlite3_config() is used to
  supply a large chunk of memory from which all memory allocations are
  taken.
  The MEMSYS3 memory allocator uses a hybrid allocation algorithm
  patterned after dlmalloc().   Only one of SQLITE_ENABLE_MEMSYS3 and
  SQLITE_ENABLE_MEMSYS5 may be enabled at once.

SQLITE_ENABLE_MEMSYS5
  This option includes code in SQLite that implements an alternative
  memory allocator.  This alternative memory allocator is only engaged
  when the SQLITE_CONFIG_HEAP option to sqlite3_config() is used to
  supply a large chunk of memory from which all memory allocations are
  taken.
  The MEMSYS5 module rounds all allocations up to the next power
  of two and uses a first-fit, buddy-allocator algorithm
  that provides strong guarantees against fragmentation and breakdown
  subject to certain operating constraints.

SQLITE_ENABLE_NORMALIZE
  This option includes the sqlite3_normalized_sql() API.

SQLITE_ENABLE_NULL_TRIM
  This option enables an optimization that omits NULL columns at
  the ends of rows, for a space savings on disk.
  
  Databases generated with this option enabled are not readable
  by SQLite version 3.1.6 (2005-03-17) and earlier.  Also,
  databases generated with this option enabled are prone to
  triggering the
  e6e962d6b0f06f46
  bug in the sqlite3_blob_reopen() interface.  For those reasons,
  this optimization is disabled by default.  However, this optimization
  may be enabled by default in a future release of SQLite.

SQLITE_ENABLE_OFFSET_SQL_FUNC
  This option enables support for the sqlite_offset(X) SQL function.
  
  The sqlite_offset(X) SQL function requires a new interface on the
  B-tree storage engine, a new opcode in the virtual machine that
  runs SQL statements, and a new conditional in a critical path of the
  code generator.  To avoid that overhead in applications that do not
  need the utility of sqlite_offset(X), the function is disabled by
  default.

SQLITE_ENABLE_PREUPDATE_HOOK
  This option enables
  several new APIs that provide callbacks
  prior to any change to a rowid table.  The callbacks can be used
  to record the state of the row before the change occurs.
  The action of the preupdate hook is similar to the
  update hook except that the callback is
  invoked before the change, not afterwards, and the preupdate
  hook interfaces are omitted unless this compile-time option is
  used.
  The preupdate hook interfaces were originally added to
  support the session extension.

SQLITE_ENABLE_QPSG
  This option causes the query planner stability guarantee (QPSG) to
  be on by default.  Normally the QPSG is off and must be activated
  at run-time using the SQLITE_DBCONFIG_ENABLE_QPSG option to the
  sqlite3_db_config() interface.

SQLITE_ENABLE_RBU
  Enable the code the implements the RBU extension.

SQLITE_ENABLE_RTREE
  This option causes SQLite to include support for the
  R*Tree index extension.

SQLITE_ENABLE_SESSION
  This option enables the session extension.

SQLITE_ENABLE_SNAPSHOT
  This option enables the code to support the sqlite3_snapshot object
  and its related interfaces:
  
   sqlite3_snapshot_get() (constructor)
   sqlite3_snapshot_free() (destructor)
   sqlite3_snapshot_open()
   sqlite3_snapshot_cmp()
   sqlite3_snapshot_recover()
  

SQLITE_ENABLE_SORTER_REFERENCES
  This option activates an optimization that reduces the memory required
  by the sorter at the cost of doing additional B-tree lookups after
  the sort has occurred.
  
  The default sorting procedure is to gather all information that will
  ultimately be output into a "record" and pass that complete record
  to the sorter.  But in some cases, for example if some of the output
  columns consists of large BLOB values, the size of the each record
  can be large, which means that the sorter has to either use more memory,
  and/or write more content to temporary storage.
  
  When SQLITE_ENABLE_SORTER_REFERENCES is enabled, the records passed to
  the sorter often contain only a ROWID value.  Such records are much
  smaller.  This means the sorter has much less "payload" to deal with and
  can run faster.  After sorting has occurred, the ROWID is used to look up
  the output column values in the original table.  That requires another
  search into the table, and could potentially result in a slowdown.  Or,
  it might be a performance win, depending on how large the values are.
  
  Even when the SQLITE_ENABLE_SORTER_REFERENCES compile-time option is on,
  sorter references are still disabled by default.  To use sorter references,
  the application must set a sorter reference size threshold using the
  sqlite3_config(SQLITE_CONFIG_SORTERREF_SIZE) interface at start-time.
  
  Because the SQLite developers do not know whether the
  SQLITE_ENABLE_SORTER_REFERENCES option will help or hurt performance,
  it is disabled by default at this time (2018-05-04).  It might be enabled
  by default in some future release, depending on what is learned about its
  impact on performance.

SQLITE_ENABLE_STMT_SCANSTATUS
  This option enables the sqlite3_stmt_scanstatus() and
  sqlite3_stmt_scanstatus_v2() interfaces.  Those
  interfaces are normally omitted from the build
  because they imposes a performance penalty, even on statements that
  do not use the feature.

SQLITE_ENABLE_STMTVTAB
  This compile-time option enables the SQLITE_STMT virtual table logic.

SQLITE_RTREE_INT_ONLY
  This compile-time option is deprecated and untested.

SQLITE_ENABLE_SQLLOG
  This option enables extra code (especially the SQLITE_CONFIG_SQLLOG
  option to sqlite3_config()) that can be used to create logs of all
  SQLite processing performed by an application.  These logs can be useful
  in doing off-line analysis of the behavior of an application, and especially
  for performance analysis.  In order for the SQLITE_ENABLE_SQLLOG option to
  be useful, some extra code is required.  The
  "test_sqllog.c"
  source code
  file in the SQLite source tree is a working example of the required extra
  code.  On unix and windows systems, a developer can append the text of the
  "test_sqllog.c" source code file to the end of an "sqlite3.c" amalgamation,
  recompile the application using the -DSQLITE_ENABLE_SQLLOG option, then
  control logging using environment variables.  See the header comment on
  the "test_sqllog.c" source file for additional detail.

SQLITE_ENABLE_STAT2
  This option used to cause the ANALYZE command to collect
  index histogram data in the sqlite_stat2 table.  But that
  functionality was superceded by SQLITE_ENABLE_STAT3 as of
  SQLite version 3.7.9 (2011-11-01).
  The SQLITE_ENABLE_STAT2 compile-time option
  is now a no-op.

SQLITE_ENABLE_STAT3
  This option used to cause the ANALYZE command to collect
  index histogram data in the sqlite_stat3 table.  But that
  functionality was superceded by SQLITE_ENABLE_STAT4 as of
  SQLite version 3.8.1 (2013-10-17).
  The SQLITE_ENABLE_STAT3 compile-time option
  continued to be supported through version 3.29.0 (2019-07-10)
  but has now become a no-op.
  

SQLITE_ENABLE_STAT4
  This option adds additional logic to the ANALYZE command and to
  the query planner that can help SQLite to chose a better query plan
  under certain situations.  The ANALYZE command is enhanced to collect
  histogram data from all columns of every index and store that data
  in the sqlite_stat4 table.  The query planner will then use the
  histogram data to help it make better index choices.  The downside of
  this compile-time option is that it violates the
  query planner stability guarantee making it more difficult to ensure
  consistent performance in mass-produced applications.
  
  SQLITE_ENABLE_STAT4 is an enhancement of SQLITE_ENABLE_STAT3.  STAT3
  only recorded histogram data for the left-most column of each index
  whereas the STAT4 enhancement records histogram data from all columns
  of each index.
  The SQLITE_ENABLE_STAT3 compile-time option has become a no-op.

SQLITE_ENABLE_TREE_EXPLAIN
  This compile-time option is no longer used.

SQLITE_ENABLE_UPDATE_DELETE_LIMIT
  This option enables an optional ORDER BY and LIMIT clause on
  UPDATE and DELETE statements.

  If this option is defined, then it must also be
  defined when using the Lemon parser generator tool to generate a parse.c
  file. Because of this, this option may only be used when the library is built
  from source, not from the amalgamation or from the collection of
  pre-packaged C files provided for non-Unix like platforms on the website.
  

SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION
  When the SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION compile-time option is
  activated, SQLite will suppress "unknown function" errors when running
  an EXPLAIN or EXPLAIN QUERY PLAN.  Instead of throwing an error,
  SQLite will insert a substitute no-op function named "unknown()".
  The substitution of "unknown()" in place of unrecognized functions
  only occurs on EXPLAIN and EXPLAIN QUERY PLAN, not on ordinary
  statements.
  
  When used in the command-line shell, the
  SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION feature allows SQL text that contains
  application-defined functions to be pasted into the shell for
  analysis and debugging without having to create and load an
  extension that implements the application-defined functions.

SQLITE_ENABLE_UNLOCK_NOTIFY
  This option enables the sqlite3_unlock_notify() interface and
  its associated functionality.  See the documentation titled
  Using the SQLite Unlock Notification Feature for additional
  information.

SQLITE_INTROSPECTION_PRAGMAS
  This option is obsolete.  It used to enable some extra
  some extra PRAGMA statements such as
  PRAGMA function_list, PRAGMA module_list, and
  PRAGMA pragma_list, but those pragmas are now all
  enabled by default.  See SQLITE_OMIT_INTROSPECTION_PRAGMAS.

SQLITE_SOUNDEX
  This option enables the soundex() SQL function.

SQLITE_USE_ALLOCA
  If this option is enabled, then the alloca() memory allocator will be
  used in a few situations where it is appropriate.  This results in a slightly
  smaller and faster binary.  The SQLITE_USE_ALLOCA compile-time only
  works, of course, on systems that support alloca().

SQLITE_USE_FCNTL_TRACE
  This option causes SQLite to issue extra SQLITE_FCNTL_TRACE file controls
  to provide supplementary information to the VFS.  The "vfslog.c" extension
  makes use of this to provide enhanced logs of VFS activity.

SQLITE_USE_SEH
  This option enabled Structured Exception Handling (SEH) on Windows builds.
  SEH is a Windows-specific technique for catching exceptions raised while
  accessing a memory-mapped file.  SEH is used to intercept errors that might
  occur while accessing the memory-mapped shm file that are part of
  WAL mode processing.  If the operating system raised errors while SQLite
  is trying to access the shm file, this option causes those errors to be
  caught and dealt with by SQLite, rather than aborting the whole process.
  
  This option only works when compiling on Windows using MSVC.

SQLITE_HAVE_ZLIB
  This option causes some extensions to link against the
  zlib compression library.
  
  This option has no effect on the SQLite core.  It is only used by extensions.
  This is option is necessary for the compression and decompression
  functions that are part of SQL Archive support in the
  command-line shell.
  
  When compiling with this option, it will normally
  be necessary to add a linker option to include the zlib library in the
  build.  Normal this option is "-lz" but might be different on different
  systems.
  
  When building with MSVC on Windows systems, one can put the zlib source
  code in the compat/zlib subdirectory of the source tree and then add
  the USE_ZLIB=1 option to the nmake command to cause the Makefile.msc
  to automatically build and use an appropriate zlib library implementation.

YYTRACKMAXSTACKDEPTH
  This option causes the LALR(1) parser stack depth to be tracked
  and reported using the sqlite3_status(SQLITE_STATUS_PARSER_STACK,...)
  interface.  SQLite's LALR(1) parser has a fixed stack depth
  (determined at compile-time using the YYSTACKDEPTH options).
  This option can be used to help determine if an application is
  getting close to exceeding the maximum LALR(1) stack depth.


c2: Compile-time Options
c3: 8. Options To Disable Features Normally Turned On
c5: compile.html#_options_to_disable_features_normally_turned_on
c4:
SQLITE_DISABLE_LFS
  If this C-preprocessor macro is defined, large file support
  is disabled.

SQLITE_DISABLE_DIRSYNC
  If this C-preprocessor macro is defined, directory syncs
  are disabled.  SQLite typically attempts to sync the parent
  directory when a file is deleted to ensure the directory
  entries are updated immediately on disk.

SQLITE_DISABLE_FTS3_UNICODE
  If this C-preprocessor macro is defined, the unicode61 tokenizer
  in FTS3 is omitted from the build and is unavailable to
  applications.

SQLITE_DISABLE_FTS4_DEFERRED
  If this C-preprocessor macro disables the "deferred token" optimization
  in FTS4.  The "deferred token" optimization avoids loading massive
  posting lists for terms that are in most documents of the collection
  and instead simply scans for those tokens in the document source.  FTS4
  should get exactly the same answer both with and without this optimization.

SQLITE_DISABLE_INTRINSIC
  This option disables the use of compiler-specific built-in functions
  such as __builtin_bswap32() and __builtin_add_overflow() in GCC and Clang,
  or _byteswap_ulong() and _ReadWriteBarrier() with MSVC.


c2: Compile-time Options
c3: 9. Options To Omit Features
c5: compile.html#_options_to_omit_features
c4:
The following options can be used to
reduce the size of the compiled library
by omitting unused features. This is probably only useful
in embedded systems where space is especially tight, as even with all
features included the SQLite library is relatively small. Don't forget
to tell your compiler to optimize for binary size! (the -Os option if
using GCC).  Telling your compiler to optimize for size usually has
a much larger impact on library footprint than employing any of these
compile-time options.  You should also verify that
debugging options are disabled.

The macros in this section do not require values. The following
compilation switches all have the same effect:
-DSQLITE_OMIT_ALTERTABLE
-DSQLITE_OMIT_ALTERTABLE=1
-DSQLITE_OMIT_ALTERTABLE=0


If any of these options are defined, then the same set of SQLITE_OMIT_*
options must also be defined when using the Lemon parser generator
tool to generate the
parse.c file and when compiling the 'mkkeywordhash' tool which generates
the keywordhash.h file.
Because of this, these options may only be used when the library is built
from canonical source, not from the amalgamation.
Some SQLITE_OMIT_* options might work, or appear to work, when used with
the amalgamation.  But this is not guaranteed.  In general, always compile
from canonical sources in order to take advantage of SQLITE_OMIT_* options.



Important Note: The SQLITE_OMIT_* options may not work with the
amalgamation.  SQLITE_OMIT_* compile-time
options usually work correctly only when SQLite is built from canonical
source files.




Special versions of the SQLite amalgamation that do work with a
predetermined set of SQLITE_OMIT_* options can be generated.  To do so,
make a copy of the Makefile.linux-gcc makefile template in the canonical
source code distribution.  Change the name of your copy to simply "Makefile".
Then edit "Makefile" to set up appropriate compile-time options.  Then
type:
make clean; make sqlite3.c

The resulting "sqlite3.c" amalgamation code file (and its associated
header file "sqlite3.h") can then be moved to a non-unix platform
for final compilation using a native compiler.

The SQLITE_OMIT_* options are unsupported.  By this we mean that
an SQLITE_OMIT_* option that omits code from the build in the current
release might become a no-op in the next release.  Or the other way around:
an SQLITE_OMIT_* that is a no-op in the current release might cause code
to be excluded in the next release.  Also, not all SQLITE_OMIT_* options
are tested.  Some SQLITE_OMIT_* options might cause SQLite to malfunction
and/or provide incorrect answers.


Important Note:
The SQLITE_OMIT_* compile-time options are mostly unsupported.


The following are the available OMIT options:


SQLITE_OMIT_ALTERTABLE
  When this option is defined, the
  ALTER TABLE command is not included in the
  library. Executing an ALTER TABLE statement causes a parse error.

SQLITE_OMIT_ANALYZE
  When this option is defined, the ANALYZE command is omitted from
  the build.

SQLITE_OMIT_ATTACH
  When this option is defined, the ATTACH and DETACH commands are
  omitted from the build.

SQLITE_OMIT_AUTHORIZATION
  Defining this option omits the authorization callback feature from the
  library. The sqlite3_set_authorizer() API function is not present
  in the library.

SQLITE_OMIT_AUTOINCREMENT
  This option is omits the AUTOINCREMENT feature.
  When this is macro is defined, columns declared as
  "INTEGER PRIMARY KEY AUTOINCREMENT"
  behave in the same way as columns declared as "INTEGER PRIMARY KEY" when a
  NULL is inserted. The sqlite_sequence system table is neither created, nor
  respected if it already exists.

SQLITE_OMIT_AUTOINIT
  For backwards compatibility with older versions of SQLite that lack
  the sqlite3_initialize() interface, the sqlite3_initialize() interface
  is called automatically upon entry to certain key interfaces such as
  sqlite3_open(), sqlite3_vfs_register(), and sqlite3_mprintf().
  The overhead of invoking sqlite3_initialize() automatically in this
  way may be omitted by building SQLite with the SQLITE_OMIT_AUTOINIT
  C-preprocessor macro.  When built using SQLITE_OMIT_AUTOINIT, SQLite
  will not automatically initialize itself and the application is required
  to invoke sqlite3_initialize() directly prior to beginning use of the
  SQLite library.

SQLITE_OMIT_AUTOMATIC_INDEX
  This option is used to omit the
  automatic indexing functionality.
  See also: SQLITE_DEFAULT_AUTOMATIC_INDEX.

SQLITE_OMIT_AUTORESET
  By default, the sqlite3_step() interface will automatically invoke
  sqlite3_reset() to reset the prepared statement if necessary.  This
  compile-time option changes that behavior so that sqlite3_step() will
  return SQLITE_MISUSE if it called again after returning anything other
  than SQLITE_ROW, SQLITE_BUSY, or SQLITE_LOCKED unless there was an
  intervening call to sqlite3_reset().

  In SQLite version 3.6.23.1 (2010-03-26)
  and earlier, sqlite3_step() used to always
  return SQLITE_MISUSE if it was invoked again after returning anything
  other than SQLITE_ROW without an intervening call to sqlite3_reset().
  This caused problems on some poorly written smartphone applications which
  did not correctly handle the SQLITE_LOCKED and SQLITE_BUSY error
  returns.  Rather than fix the many defective smartphone applications,
  the behavior of SQLite was changed in 3.6.23.2 to automatically reset
  the prepared statement.  But that changed caused issues in other
  improperly implemented applications that were actually looking
  for an SQLITE_MISUSE return to terminate their query loops.  (Anytime
  an application gets an SQLITE_MISUSE error code from SQLite, that means the
  application is misusing the SQLite interface and is thus incorrectly
  implemented.)  The SQLITE_OMIT_AUTORESET interface was added to SQLite
  version 3.7.5 (2011-02-01) in an effort to get all of the (broken)
  applications to work again without having to actually fix the applications.

SQLITE_OMIT_AUTOVACUUM
  If this option is defined, the library cannot create or write to
  databases that support auto_vacuum.
  Executing a PRAGMA auto_vacuum statement is not an error
  (since unknown PRAGMAs are silently ignored), but does not return a value
  or modify the auto-vacuum flag in the database file. If a database that
  supports auto-vacuum is opened by a library compiled with this option, it
  is automatically opened in read-only mode.

SQLITE_OMIT_BETWEEN_OPTIMIZATION
  This option disables the use of indices with WHERE clause terms
  that employ the BETWEEN operator.

SQLITE_OMIT_BLOB_LITERAL
  When this option is defined, it is not possible to specify a blob in
  an SQL statement using the X'ABCD' syntax.

SQLITE_OMIT_BTREECOUNT
  This option is no longer used for anything.  It is a no-op.

SQLITE_OMIT_BUILTIN_TEST
  This compile-time option has been renamed to SQLITE_UNTESTABLE.

SQLITE_OMIT_CASE_SENSITIVE_LIKE_PRAGMA
  This compile-time option disables the PRAGMA case_sensitive_like
  command.

SQLITE_OMIT_CAST
  This option causes SQLite to omit support for the CAST operator.

SQLITE_OMIT_CHECK
  This option causes SQLite to omit support for CHECK constraints.
  The parser will still accept CHECK constraints in SQL statements,
  they will just not be enforced.

SQLITE_OMIT_COMPILEOPTION_DIAGS
  This option is used to omit the compile-time option diagnostics available
  in SQLite, including the sqlite3_compileoption_used() and
  sqlite3_compileoption_get() C/C++ functions, the
  sqlite_compileoption_used() and sqlite_compileoption_get() SQL functions,
  and the compile_options pragma.

SQLITE_OMIT_COMPLETE
  This option causes the sqlite3_complete() and sqlite3_complete16()
  interfaces to be omitted.

SQLITE_OMIT_COMPOUND_SELECT
  This option is used to omit the compound SELECT functionality.
  SELECT statements that use the
  UNION, UNION ALL, INTERSECT or EXCEPT compound SELECT operators will
  cause a parse error.

  An INSERT statement with multiple values in the VALUES clause is
  implemented internally as a compound SELECT.  Hence, this option also
  disables the ability to insert more than a single row using an
  INSERT INTO ... VALUES ... statement.

SQLITE_OMIT_CTE
  This option causes support for common table expressions to be omitted.

SQLITE_OMIT_DATETIME_FUNCS
  If this option is defined, SQLite's built-in date and time manipulation
  functions are omitted. Specifically, the SQL functions julianday(), date(),
  time(), datetime() and strftime() are not available. The default column
  values CURRENT_TIME, CURRENT_DATE and CURRENT_TIMESTAMP are still available.

SQLITE_OMIT_DECLTYPE
  This option causes SQLite to omit support for the
  sqlite3_column_decltype() and sqlite3_column_decltype16()
  interfaces.

SQLITE_OMIT_DEPRECATED
  This option causes SQLite to omit support for interfaces
  marked as deprecated.  This includes
  sqlite3_aggregate_count(),
  sqlite3_expired(),
  sqlite3_transfer_bindings(),
  sqlite3_global_recover(),
  sqlite3_thread_cleanup() and
  sqlite3_memory_alarm() interfaces and
  PRAGMA statements PRAGMA count_changes,
  PRAGMA data_store_directory,
  PRAGMA default_cache_size,
  PRAGMA empty_result_callbacks,
  PRAGMA full_column_names,
  PRAGMA short_column_names, and
  PRAGMA temp_store_directory.

SQLITE_OMIT_DESERIALIZE
  This option causes the
  sqlite3_serialize() and sqlite3_deserialize()
  interfaces to be omitted from the build.

SQLITE_OMIT_DISKIO
  This option omits all support for writing to the disk and forces
  databases to exist in memory only.  This option has not been
  maintained and probably does not work with newer versions of SQLite.

SQLITE_OMIT_EXPLAIN
  Defining this option causes the EXPLAIN command to be omitted from the
  library. Attempting to execute an EXPLAIN statement will cause a parse
  error.

SQLITE_OMIT_FLAG_PRAGMAS
  This option omits support for a subset of PRAGMA commands that
  query and set boolean properties.

SQLITE_OMIT_FLOATING_POINT
  This option is used to omit floating-point number support from the SQLite
  library. When specified, specifying a floating point number as a literal
  (i.e. "1.01") results in a parse error.

  In the future, this option may also disable other floating point
  functionality, for example the sqlite3_result_double(),
  sqlite3_bind_double(), sqlite3_value_double() and
  sqlite3_column_double() API functions.
  

SQLITE_OMIT_FOREIGN_KEY
  If this option is defined, then foreign key constraint syntax is
  not recognized.

SQLITE_OMIT_GENERATED_COLUMNS
  If this option is defined, then generated column syntax is
  not recognized.

SQLITE_OMIT_GET_TABLE
  This option causes support for sqlite3_get_table() and
  sqlite3_free_table() to be omitted.

SQLITE_OMIT_HEX_INTEGER
  This option omits support for hexadecimal integer literals.

SQLITE_OMIT_INCRBLOB
  This option causes support for incremental BLOB I/O
  to be omitted.

SQLITE_OMIT_INTEGRITY_CHECK
  This option omits support for the integrity_check pragma.

SQLITE_OMIT_INTROSPECTION_PRAGMAS
  This option omits support for
  PRAGMA function_list, PRAGMA module_list, and
  PRAGMA pragma_list.

SQLITE_OMIT_JSON
  This option omits the JSON SQL functions from the build.

SQLITE_OMIT_LIKE_OPTIMIZATION
  This option disables the ability of SQLite to use indices to help
  resolve LIKE and GLOB operators in a WHERE clause.

SQLITE_OMIT_LOAD_EXTENSION
  This option omits the entire extension loading mechanism from
  SQLite, including sqlite3_enable_load_extension() and
  sqlite3_load_extension() interfaces.

SQLITE_OMIT_LOCALTIME
  This option omits the "localtime" modifier from the date and time
  functions.  This option is sometimes useful when trying to compile
  the date and time functions on a platform that does not support the
  concept of local time.

SQLITE_OMIT_LOOKASIDE
  This option omits the lookaside memory allocator.

SQLITE_OMIT_MEMORYDB
  When this is defined, the library does not respect the special database
  name ":memory:" (normally used to create an in-memory database). If
  ":memory:" is passed to sqlite3_open(), sqlite3_open16(), or
  sqlite3_open_v2(), a file with this name will be
  opened or created.

SQLITE_OMIT_OR_OPTIMIZATION
  This option disables the ability of SQLite to use an index together
  with terms of a WHERE clause connected by the OR operator.

SQLITE_OMIT_PAGER_PRAGMAS
  Defining this option omits pragmas related to the pager subsystem from
  the build.

SQLITE_OMIT_PRAGMA
  This option is used to omit the PRAGMA command
  from the library. Note that it is useful to define the macros that omit
  specific pragmas in addition to this, as they may also remove supporting code
  in other sub-systems. This macro removes the PRAGMA command only.

SQLITE_OMIT_PROGRESS_CALLBACK
  This option may be defined to omit the capability to issue "progress"
  callbacks during long-running SQL statements. The
  sqlite3_progress_handler()
  API function is not present in the library.

SQLITE_OMIT_QUICKBALANCE
  This option omits an alternative, faster B-Tree balancing routine.
  Using this option makes SQLite slightly smaller at the expense of
  making it run slightly slower.

SQLITE_OMIT_REINDEX
  When this option is defined, the REINDEX
  command is not included in the library.
  Executing a REINDEX statement causes
  a parse error.

SQLITE_OMIT_SCHEMA_PRAGMAS
  Defining this option omits pragmas for querying the database schema from
  the build.

SQLITE_OMIT_SCHEMA_VERSION_PRAGMAS
  Defining this option omits pragmas for querying and modifying the
  database schema version and user version from the build. Specifically, the
  schema_version and user_version PRAGMAs are omitted.

SQLITE_OMIT_SHARED_CACHE
  This option builds SQLite without support for shared cache mode.
  The sqlite3_enable_shared_cache() is omitted along with a fair
  amount of logic within the B-Tree subsystem associated with shared
  cache management.

  This compile-time option is recommended most applications as it
  results in improved performance and reduced library footprint.

SQLITE_OMIT_SUBQUERY
  If defined, support for sub-selects and the IN() operator are omitted.

SQLITE_OMIT_TCL_VARIABLE
  If this macro is defined, then the special "$" syntax
  used to automatically bind SQL variables to TCL variables is omitted.

SQLITE_OMIT_TEMPDB
  This option omits support for TEMP or TEMPORARY tables.

SQLITE_OMIT_TRACE
  This option omits support for the sqlite3_profile() and
  sqlite3_trace() interfaces and their associated logic.

SQLITE_OMIT_TRIGGER
  Defining this option omits support for TRIGGER objects. Neither the
  CREATE TRIGGER or DROP TRIGGER
  commands are available in this case, and attempting to execute
  either will result in a parse error.
  This option also disables enforcement of foreign key constraints,
  since the code that implements triggers and which is omitted by this
  option is also used to implement foreign key actions.

SQLITE_OMIT_TRUNCATE_OPTIMIZATION
  A default build of SQLite, if a DELETE statement has no WHERE clause
  and operates on a table with no triggers, an optimization occurs that
  causes the DELETE to occur by dropping and recreating the table.
  Dropping and recreating a table is usually much faster than deleting
  the table content row by row.  This is the "truncate optimization".

SQLITE_OMIT_UTF16
  This macro is used to omit support for UTF16 text encoding. When this is
  defined all API functions that return or accept UTF16 encoded text are
  unavailable. These functions can be identified by the fact that they end
  with '16', for example sqlite3_prepare16(), sqlite3_column_text16() and
  sqlite3_bind_text16().

SQLITE_OMIT_VACUUM
  When this option is defined, the VACUUM
  command is not included in the library.
  Executing a VACUUM statement causes
  a parse error.

SQLITE_OMIT_VIEW
  Defining this option omits support for VIEW objects. Neither the
  CREATE VIEW nor the DROP VIEW
  commands are available in this case, and
  attempting to execute either will result in a parse error.

  WARNING: If this macro is defined, it will not be possible to open a database
  for which the schema contains VIEW objects.

SQLITE_OMIT_VIRTUALTABLE
  This option omits support for the Virtual Table
  mechanism in SQLite.

SQLITE_OMIT_WAL
  This option omits the "write-ahead log" (a.k.a. "WAL") capability.

SQLITE_OMIT_WINDOWFUNC
  This option omits window functions from the build.

SQLITE_OMIT_WSD
  This option builds a version of the SQLite library that contains no
  Writable Static Data (WSD).  WSD is global variables and/or static
  variables.  Some platforms do not support WSD, and this option is necessary
  in order for SQLite to work those platforms.

  Unlike other OMIT options which make the SQLite library smaller,
  this option actually increases the size of SQLite and makes it run
  a little slower.  Only use this option if SQLite is being built for an
  embedded target that does not support WSD.

SQLITE_OMIT_XFER_OPT
  This option omits support for optimizations that help statements
  of the form "INSERT INTO ... SELECT ..." run faster.

SQLITE_UNTESTABLE
  A standard SQLite build includes a small amount of logic associated
  with sqlite3_test_control() to exercise
  parts of the SQLite core that are otherwise difficult to validate.
  This compile-time option omits that extra testing logic.  This
  compile-time option was called "SQLITE_OMIT_BUILTIN_TEST" prior
  to SQLite version 3.16.0 (2017-01-02).  The name was changed
  to better describe the implications of using it.
  
  Setting this compile-time option prevents SQLite from being fully
  testable.  Branch test coverage drops from 100% down to about 95%.
  
  SQLite developers follow the NASA principle of
  "fly what you test and test what you fly".  This principle is violated
  if this option is enabled for delivery but disabled for testing.
  But if this option is enabled during testing, not all branches are
  reachable.  Therefore, the use of this compile-time option is discouraged.

SQLITE_ZERO_MALLOC
  This option omits both the default memory allocator and the
  debugging memory allocator from the build and substitutes a stub
  memory allocator that always fails.  SQLite will not run with this
  stub memory allocator since it will be unable to allocate memory.  But
  this stub can be replaced at start-time using
  sqlite3_config(SQLITE_CONFIG_MALLOC,...) or
  sqlite3_config(SQLITE_CONFIG_HEAP,...).
  So the net effect of this compile-time option is that it allows SQLite
  to be compiled and linked against a system library that does not support
  malloc(), free(), and/or realloc().


c2: Compile-time Options
c3: 10. Analysis and Debugging Options
c5: compile.html#_analysis_and_debugging_options
c4:
SQLITE_DEBUG
  The SQLite source code contains literally thousands of assert() statements
  used to verify internal assumptions and subroutine preconditions and
  postconditions.  These assert() statements are normally turned off
  (they generate no code) since turning them on makes SQLite run approximately
  three times slower.  But for testing and analysis, it is useful to turn
  the assert() statements on.  The SQLITE_DEBUG compile-time option does this.
  SQLITE_DEBUG also enables some other debugging features, such as
  special PRAGMA statements that turn on tracing and listing features
  used for troubleshooting and analysis of the VDBE and code generator.

SQLITE_MEMDEBUG
  The SQLITE_MEMDEBUG option causes an instrumented
  debugging memory allocator
  to be used as the default memory allocator within SQLite.  The
  instrumented memory allocator checks for misuse of dynamically allocated
  memory.  Examples of misuse include using memory after it is freed,
  writing off the ends of a memory allocation, freeing memory not previously
  obtained from the memory allocator, or failing to initialize newly
  allocated memory.


c2: Compile-time Options
c3: 11. Windows-Specific Options
c5: compile.html#_windows_specific_options
c4:
SQLITE_WIN32_HEAP_CREATE
  This option forces the Win32 native memory allocator, when enabled, to
  create a private heap to hold all memory allocations.

SQLITE_WIN32_MALLOC_VALIDATE
  This option forces the Win32 native memory allocator, when enabled, to
  make strategic calls into the HeapValidate() function if assert() is also
  enabled.


c2: Compile-time Options
c3: 12. Compiler Linkage and Calling Convention Control
c5: compile.html#compiler_linkage_and_calling_convention_control
c4:
The following macros specify interface details
for certain kinds of SQLite builds.  The Makefiles will normally
handle setting these macros automatically.  Application developers should
not need to worry with these macros.  The following documentation about these
macros is included for completeness.


SQLITE_API
  This macro identifies an externally visible interface for SQLite.
  This macro is sometimes set to "extern".  But the definition is
  compiler-specific.

SQLITE_APICALL
  This macro identifies the calling convention used by public interface
  routines in SQLite which accept a fixed number of arguments.
  This macro is normally defined to be nothing,
  though on Windows builds it can sometimes be set to "__cdecl" or "__stdcall".
  The "__cdecl" setting is the default, but "__stdcall" is used when SQLite
  is intended to be compiled as a Windows system library.
  
  A single function declaration should contain no more than one of
  the following:  SQLITE_APICALL, SQLITE_CDECL, or SQLITE_SYSAPI.

SQLITE_CALLBACK
  This macro specifies the calling convention used with callback pointers
  in SQLite.  This macro is normally defined to be nothing, though on Windows
  builds it can sometimes be set to "__cdecl" or "__stdcall".  The
  "__cdecl" setting is the default, but "__stdcall" is used when SQLite
  is intended to be compiled as a Windows system library.

SQLITE_CDECL
  This macro specifies the calling convention used by varargs interface
  routines in SQLite.  This macro is normally defined to be nothing,
  though on Windows builds it can sometimes be set to "__cdecl".  This
  macro is used on varargs routines and so cannot be set to "__stdcall"
  since the __stdcall calling convention does not support varargs functions.
  
  A single function declaration should contain no more than one of
  the following:  SQLITE_APICALL, SQLITE_CDECL, or SQLITE_SYSAPI.

SQLITE_EXTERN
  This macro specifies linkage for public interface variables in SQLite.
  It should normally be allowed to default to "extern".

SQLITE_STDCALL
  This macro is no longer used and is now deprecated.

SQLITE_SYSAPI
  This macro identifies the calling convention used by operating system
  interfaces for the target platform for an SQLite build.
  This macro is normally defined to be nothing,
  though on Windows builds it can sometimes be set to "__stdcall".
  
  A single function declaration should contain no more than one of
  the following:  SQLITE_APICALL, SQLITE_CDECL, or SQLITE_SYSAPI.

SQLITE_TCLAPI
  This macro specifies the calling convention used by the
  TCL library interface routines.
  This macro is not used by the SQLite core, but only by the TCL Interface
  and TCL test suite.
  This macro is normally defined to be nothing,
  though on Windows builds it can sometimes be set to "__cdecl".  This
  macro is used on TCL library interface routines which are always compiled
  as __cdecl, even on platforms that prefer to use __stdcall, so this
  macro should not be set to __stdcall unless the platform has a custom
  TCL library build that supports __stdcall.
  
  This macro may not be used in combination with any of SQLITE_APICALL,
  SQLITE_CALLBACK, SQLITE_CDECL or SQLITE_SYSAPI.


