Sqlite

Sqlite :: # (opaque)

Open SQLite databases, run statements, and decode rows.

A Db is one connection. A string literal is a Query: each ${…} in it is a Value bound as a parameter, never pasted into the SQL text. A Decoder turns a result row into a Roc value.

import pf.Sqlite
import pf.Sqlite.Decoder

decode_todo : Decoder({ id : I64, task : Str }, Sqlite.DecodeErr)
decode_todo = { id: Decoder.i64("id"), task: Decoder.str("task") }.Decoder

db = Sqlite.open!("app.db", {})?
todo = db.query_one!("SELECT id, task FROM todos WHERE id = ${Integer(id)}", decode_todo)?

Without the second import the same names are Sqlite.Decoder.i64("id") and { … }.Sqlite.Decoder.

open! : Path, OpenOptions => Try(Db, [SqliteErr(Err)])

Open a connection to the database file at path.

Every call opens a new connection, also for a path that is already open. The path ":memory:" opens a new, private in-memory database, like [open_in_memory!].

db = Sqlite.open!("app.db", {})?
readonly = Sqlite.open!("app.db", { mode: ReadOnly })?
open_in_memory! : () => Try(Db, [SqliteErr(Err)])

Open a new, private in-memory database. It is discarded when the connection is closed or released.

Value : [
    Null,
    Real(F64),
    Integer(I64),
    String(Str),
    Bytes(List(U8)),
]

A value bound to a parameter or read from a column.

OpenOptions

:= {
    mode : [ReadWriteCreate, ReadWrite, ReadOnly] ?? ReadWriteCreate,
    busy_timeout : Duration ?? Duration.seconds(5),
}

How to open a database; {} takes every default.

mode is ReadWriteCreate (create the file if it is missing), ReadWrite (the file must exist) or ReadOnly (the file must exist and every write fails with Failed({ code: ReadOnly, .. })).

busy_timeout is how long a statement waits for another connection's lock before failing with Failed({ code: Busy, .. }). Duration.zero fails at once.

Db

Sqlite.Db :: # (opaque)

A connection to one database.

The connection is closed when the last reference to the Db, and to every unfinished Db.rows stream made from it, is released. [close!] closes it earlier and reports a failure to do so.

The connection keeps the statements it has prepared, keyed by their SQL text, so running the same text again does not prepare it again. It keeps at most 32, dropping the least recently used.

to_inspect : Db -> Str

Render the connection without exposing its host handle.

close! : Db => Try({}, [SqliteErr(Err)])

Close the connection now. An open transaction is rolled back, and every unfinished Db.rows stream ends with Err(Closed).

After close!, every other method returns Err(Closed). Closing a closed connection is Ok({}).

execute! : Db, Query => Try({}, [SqliteErr(Err), Closed])

Run one statement that returns no rows, such as an INSERT, UPDATE, DELETE or CREATE TABLE.

A statement with result columns is rejected before it runs with ReturnsRows, even if it would return no row; so an INSERT … RETURNING inserts nothing. Text holding a second statement is rejected with MultipleStatements; use [execute_script!] for that. Text holding no statement at all, only white space or comments, does nothing.

db.execute!("INSERT INTO todos (task) VALUES (${String(task)})")?
execute_script! : Db, Str => Try({}, [SqliteErr(Err), Closed])

Run every statement in sql, in order, stopping at the first one that fails. The statements take no parameters, and any rows they return are discarded.

The statements are not wrapped in a transaction: those before a failing one have taken effect, unless the text itself has a BEGIN or the call is made inside [transaction!].

db.execute_script!(
    \\CREATE TABLE todos (id INTEGER PRIMARY KEY, task TEXT NOT NULL);
    \\CREATE INDEX todos_task ON todos (task);
)?
query_one! : Db, Query, Decoder(a, err) => Try(a, [SqliteErr(Err), Closed, NoRows, TooManyRows, BadRow(err)])

Run a query that returns exactly one row and decode it.

No row is Err(NoRows) and a second row is Err(TooManyRows). A row whose values the decoder rejects is Err(BadRow(err)).

count = db.query_one!("SELECT COUNT(*) FROM todos", Decoder.i64(0))?
query_all! : Db, Query, Decoder(a, err) => Try(List(a), [SqliteErr(Err), Closed, BadRow(err)])

Run a query and decode every row into a list.

The first row whose values the decoder rejects ends the query with Err(BadRow(err)); to skip such rows instead, use Db.rows.

tasks = db.query_all!("SELECT task FROM todos ORDER BY id", Decoder.str("task"))?
rows : Db, Query, Decoder(a, err) -> Stream(Try(a, [SqliteErr(Err), Closed, BadRow(err)]))

Lazily run a query, decoding one row for each item pulled.

Building the stream does nothing; the first pull prepares the statement, binds its parameters and resolves the decoder's columns. The stream ends after the last row.

A BadRow(err) item concerns one row, whose values the decoder rejected, and the stream continues with the next row. A SqliteErr or Closed item is the last one. A column the decoder names that the query does not return is Err(SqliteErr(ColumnNotFound(_))), the only item.

for! item in db.rows("SELECT id, payload FROM events ORDER BY id", decode_event) {
    match item {
        Ok(event) if event.id > last => break
        Ok(event) => Stdout.line!(event.payload)?
        Err(BadRow(_)) => {}
        Err(SqliteErr(err)) => { return Err(SqliteErr(err)) }
        Err(Closed) => { return Err(Closed) }
    }
}

The stream holds its statement, and with it SQLite's read lock, until it has finished or is released. The connection stays usable meanwhile, also for another stream of the same query.

A stream left with break is released when the function containing the loop returns, not at the end of the loop (roc-lang/roc#12266). Give the loop a function of its own if the lock has to be gone before the next statement, such as a DROP TABLE.

transaction! : Db, (() => Try(a, [SqliteErr(Err), Closed, ..errs])) => Try(a, [SqliteErr(Err), Closed, ..errs])

Query

Sqlite.Query :: # (opaque)

SQL text and the values of its parameters.

A string literal is a Query. Each ${…} in it must be a Value and becomes a bound parameter (?1, ?2, …), so the value is never parsed as SQL:

db.execute!("UPDATE todos SET status = ${String(status)} WHERE id = ${Integer(id)}")?

A ${…} inside '…', "…" or a comment would be text, not a parameter, so such a literal is rejected at compile time; write name = ${String(name)}, not name = '${String(name)}'.

from_quote : Str -> Try(Query, [BadQuotedBytes(Str)])

A string literal without interpolation: SQL with no parameters.

new : Str, List(Value) -> Query

SQL assembled at run time, with the values for its ? markers in order. A count that differs from the statement's number of parameters is WrongParamCount when the query is run.

marks = Str.join_with(ids.map(|_| "?"), ", ")
query = Query.new("SELECT task FROM todos WHERE id IN (${marks})", ids.map(|id| Integer(id)))

Column

:= [Name(Str), Index(U64)]

A result column, by name or by position counted from zero. A string literal is a name and a number literal is a position: Decoder.i64("id"), Decoder.i64(0).

A name must match the column's name in the result exactly; give an expression a name with AS. Where two columns have the same name, the name selects the first.

ErrCode : [
    Error,
    Internal,
    Perm,
    Abort,
    Busy,
    Locked,
    NoMem,
    ReadOnly,
    Interrupt,
    Io,
    Corrupt,
    NotFound,
    Full,
    CanNotOpen,
    Protocol,
    Empty,
    Schema,
    TooBig,
    Constraint([Unique, PrimaryKey, ForeignKey, NotNull, Check, Other]),
    Mismatch,
    Misuse,
    NoLFS,
    AuthDenied,
    Format,
    OutOfRange,
    NotADatabase,
    Unknown(I64),
]

SQLite's primary result codes for a failure. A constraint violation says which kind of constraint failed.

Err

:= [
    Failed({ code : ErrCode, extended_code : I64, message : Str }),
    ColumnNotFound(Column),
    WrongParamCount({ expected : U64, given : U64 }),
    ReturnsRows,
    MultipleStatements,
]

Why a database call failed.

Failed is an error reported by SQLite: its primary result code, its extended result code and its message. The other variants mean the call does not fit the statement, and are found before anything runs:

  • ColumnNotFound: a decoder reads a column the query does not return.
  • WrongParamCount: the statement has expected parameters and the Query carries given values.
  • ReturnsRows: a statement given to [Db.execute!] has result columns.
  • MultipleStatements: the text holds more than one statement.
to_str : Err -> Str

Describe the error for display.

DecodeErr : [
    UnexpectedType({ column : Str, found : [Null, Integer, Real, String, Bytes] }),
    BadUtf8({ problem : Utf8Problem, index : U64 }),
]

What the leaf decoders report: a column holding another type than the one asked for, or text that is not valid UTF-8.

Decoder

Sqlite.Decoder(a, err) :: # (opaque)

How to turn a result row into an a, or fail with an err.

Leaf decoders read one column. Records of decoders combine with the record-builder suffix:

decode_todo : Decoder({ id : I64, task : Str }, Sqlite.DecodeErr)
decode_todo = { id: Decoder.i64("id"), task: Decoder.str("task") }.Decoder

A decoder's columns are looked up once per query, before its first row is read; a column that is missing is SqliteErr(ColumnNotFound(_)). A value the decoder rejects fails that row only, as BadRow(err).

SQLite stores a value with its own type, whatever the column was declared as, and the leaf decoders do not convert: i64 accepts only an integer.

map2 : Decoder(a, err), Decoder(b, err), (a, b -> c) -> Decoder(c, err)

Combine two decoders; this is what the record-builder suffix { … }.Decoder calls.

map : Decoder(a, err), (a -> b) -> Decoder(b, err)

Transform the decoded value.

map_try : Decoder(a, err), (a -> Try(b, err)) -> Decoder(b, err)

Transform the decoded value with a function that can fail; its error fails the row, beside the decoder's own errors.

Decoder.i64("age").map_try(|n| if n >= 0 { Ok(n) } else { Err(NegativeAge(n)) })
nullable : Decoder(a, err) -> Decoder(Try(a, [Null]), err)

Accept SQL NULL: the result is Err(Null) when every column the decoder reads is NULL, and Ok of the decoded value otherwise.

i64 : Column -> Decoder(I64, [UnexpectedType({ column : Str, found : [Null, Integer, Real, String, Bytes] })])
f64 : Column -> Decoder(F64, [UnexpectedType({ column : Str, found : [Null, Integer, Real, String, Bytes] })])

Decode a floating-point column. An integer is not converted.

bool : Column -> Decoder(Bool, [UnexpectedType({ column : Str, found : [Null, Integer, Real, String, Bytes] })])

Decode an integer column as a boolean, the way SQLite does: zero is False and any other integer is True.

bytes : Column -> Decoder(List(U8), [UnexpectedType({ column : Str, found : [Null, Integer, Real, String, Bytes] })])

Decode a BLOB column.

str : Column -> Decoder(Str, [UnexpectedType({ column : Str, found : [Null, Integer, Real, String, Bytes] }), BadUtf8({ problem : Utf8Problem, index : U64 })])

Decode a text column. Text that is not valid UTF-8 is BadUtf8.

value : Column -> Decoder(Value, [BadUtf8({ problem : Utf8Problem, index : U64 })])

Decode a column of any type, keeping its type as a tag.

row : Decoder(Row, [BadUtf8({ problem : Utf8Problem, index : U64 })])

Decode every column of the row, with the column names.