Mentat

Mentat

Mentat is a Datomic-like database: an entity-attribute-value store with an immutable transaction log, a Datalog query language, and a declarative pull API, all expressed in EDN. One repository builds three backends that share the same Datalog/EDN front-end:

Embedded (mentat) — a Rust library store on SQLite. cargo build, no external services. Drop it into an application the way you would SQLite. Also a SQLite loadable extension (mentat_sqlite_ext) so any SQLite host can call mentat from SQL.
PostgreSQL extension (pg_mentat) — the same data model implemented inside PostgreSQL via pgrx, reached through SQL functions. Plus mentatd, an HTTP/WebSocket server that fronts a pg_mentat database.
DuckDB extension (mentat_duckdb) — loads into DuckDB (LOAD mentat;) and exposes the embedded store as SQL functions, so Datalog results join against native DuckDB tables.

All three expose the same four SQL functions:

Function	Does	PostgreSQL	SQLite ext	DuckDB
`edn_t`	transact EDN, return a JSON tx-report	`edn_t(edn)`	`edn_t(db_path, edn)`	`edn_t(db_path, edn)`
`edn_q`	run a Datalog query	`edn_q(query, inputs jsonb)` → JSONB	`edn_q(db_path, query, inputs)` → JSON text	`edn_q(db_path, query, inputs)` → rows (table function)
`edn_pull`	pull a pattern for one entity, as JSON	`edn_pull(pattern, entity)`	`edn_pull(db_path, pattern, entity)`	`edn_pull(db_path, pattern, entity)`
`edn_eval`	run a sandboxed mino script	`edn_eval(script)` (`script` feature)	`edn_eval(db_path, script)`	`edn_eval(db_path, script)`

inputs is the same JSON everywhere: {"inputs": [...]} binds the query’s :in forms positionally (scalars, [?x ...] collections, [?a ?b] tuples, [[?a ?b]] relations, in any mix), {"asOf": tx} / {"since": tx} query the database as of or since a transaction, and {} means none. The SQLite and DuckDB functions take the mentat store’s file path first; that store is separate from whatever database the host has open.

All backends parse queries, transactions, and schema with one copy of the edn, core-traits, and core crates, so a query means the same thing on every backend. They descend from Mozilla’s Project Mentat; the PostgreSQL backend was formerly the separate pg_mentat project, now merged here.

Datomic is a trademark of its owner; this project is an independent reimplementation of the model, not affiliated with or endorsed by Datomic.

New to EDN or Datalog?

Mentat speaks EDN (Clojure’s data notation) and answers questions in Datalog, the query language Datomic made popular. If you know nothing about EDN or Datalog, these are good starting points:

Learn Datalog Today — an interactive, exercise-driven Datalog tutorial.
An introduction to Datalog — a short, readable overview from Michelin’s engineering blog.
Datalog: Biting the Silver Bullet — Norbert Wójtowicz at GeeCON 2018.

Documentation

The full manual (architecture, Datalog reference, pull API, time travel, the PostgreSQL cookbook, operations, and the scripting layer) is an mdBook under docs/. Build it with mdbook build docs and open docs/book/index.html, or read the sources in docs/src/.

Architecture — the front-end/backend split, the crate map, and which features live on which backend.
Scripting — the mino mentat.store/* surface and edn_eval, with the full security model.

Embedded store (`mentat`, on SQLite)

Build and test the embedded side — a plain cargo build needs no pg_config, libclang, or PostgreSQL:

cargo build            # builds the SQLite side (workspace default members)
cargo test --workspace # runs the embedded test suite

Add it to a Rust project:

[dependencies]
mentat = { git = "https://codeberg.org/gregburd/mentat" }

Open a store, define an attribute, transact a fact, and query it:

use mentat::{Store, QueryResults, Queryable};

fn main() -> mentat::Result<()> {
    // A file path, or "" for an in-memory store.
    let mut store = Store::open("example.db")?;

    // Define schema, then assert a fact. `transact` takes EDN text.
    store.transact(r#"[
        {:db/ident       :person/name
         :db/valueType   :db.type/string
         :db/cardinality :db.cardinality/one}
    ]"#)?;
    store.transact(r#"[{:person/name "Alice"}]"#)?;

    // Query. `q_once` runs a query with optional inputs and returns a QueryOutput.
    let results = store
        .q_once(r#"[:find ?name :where [?e :person/name ?name]]"#, None)?
        .into();
    if let QueryResults::Rel(rows) = results {
        for row in rows.into_iter() {
            println!("{:?}", row); // [Alice]
        }
    }
    Ok(())
}

The embedded side also ships a C ABI (crates/sqlite/ffi) for non-Rust hosts and a CLI (crates/sqlite/cli).

PostgreSQL extension (`pg_mentat`)

pg_mentat needs a PostgreSQL install with development headers plus LLVM/clang (pgrx uses bindgen). It supports PostgreSQL 13–18; pg16 is the default feature.

Build and install

With cargo pgrx (0.17):

cargo install --locked cargo-pgrx --version 0.17.0
cargo pgrx init --pg16 $(which pg_config)     # one-time, points pgrx at your PG

cd crates/pg/pg_mentat
cargo pgrx install --release --no-default-features --features pg16

Or with Nix (no pgrx toolchain to set up by hand):

nix build .#pg_mentat-pg16     # or .#pg_mentat-pg13 … .#pg_mentat-pg18

Use it

CREATE EXTENSION pg_mentat;

-- Define a schema attribute. edn_t takes EDN transaction text.
SELECT edn_t('[
  {:db/ident       :person/name
   :db/valueType   :db.type/string
   :db/cardinality :db.cardinality/one}
]');

-- Assert a fact.
SELECT edn_t('[{:person/name "Alice"}]');

-- Query. edn_q takes an EDN query and a JSONB inputs map; returns JSONB.
SELECT edn_q(
  '[:find ?name :where [?e :person/name ?name]]',
  '{}'::jsonb
);

-- Bind :in inputs, or query the database as of an earlier transaction.
SELECT edn_q('[:find ?e :in ?name :where [?e :person/name ?name]]',
             '{"inputs": ["Alice"]}');
SELECT edn_q('[:find ?name :where [?e :person/name ?name]]', '{"asOf": 268435457}');

-- Pull all attributes for an entity (entity id 10001 here).
SELECT edn_pull('[*]', 10001);

mentat.q, mentat.t, and mentat.pull are shorter aliases for edn_q, edn_t, and edn_pull. The pre-1.9.0 names mentat_transact, mentat_query, mentat_pull, and mentat_eval still work as deprecated aliases and will be removed in a future major release; ALTER EXTENSION pg_mentat UPDATE adds the new names to an existing 1.8.0 install. As of 1.8.0 the embedded SQLite backend also does historical (as-of/since) Datalog queries, ?added history patterns, and collection/tuple/relation :in bindings. The PostgreSQL backend adds features unique to it: LISTEN/NOTIFY reactive subscriptions and integrations with pgvector, pg_trgm, PostGIS, rum, and more. See Architecture for the full feature-by-backend table.

mentatd

crates/pg/mentatd is an HTTP/WebSocket server that talks to a pg_mentat database over tokio-postgres (cargo build -p mentatd; no PostgreSQL headers needed). See the mentatd chapter.

DuckDB extension (`mentat_duckdb`)

mentat_duckdb (crates/duckdb) is a loadable DuckDB extension that embeds the mentat SQLite store and exposes it to DuckDB as SQL functions, so Datalog results can be joined against native DuckDB tables. It is built with duckdb-rs and pinned to DuckDB v1.5.5 (via the DuckDB unstable C API); the extension loads only into that DuckDB version, and bumping DuckDB means bumping the pin and rebuilding.

Build

The extension is a cdylib that needs a metadata footer, so it is built with the DuckDB extension-ci-tools Makefiles (a git submodule under crates/duckdb/):

git submodule update --init crates/duckdb/extension-ci-tools
cd crates/duckdb
make configure          # one-time: sets up the build platform + test venv
make debug              # -> build/debug/mentat.duckdb_extension
# make release for an optimized build

(mentat_duckdb is a workspace member but not a default member, so a plain cargo build never pulls the DuckDB toolchain.)

Use it

DuckDB refuses unsigned extensions unless started with -unsigned (or opened with allow_unsigned_extensions=true). With a DuckDB v1.5.5 CLI:

-- duckdb -unsigned
LOAD './build/debug/mentat.duckdb_extension';

-- Define a schema attribute and assert facts into an embedded mentat store.
-- Every function takes the store's file path as its first argument.
SELECT edn_t('/tmp/demo.mentat', '[
  {:db/ident :person/name :db/valueType :db.type/string :db/cardinality :db.cardinality/one}
]');
SELECT edn_t('/tmp/demo.mentat', '[{:person/name "Alice"} {:person/name "Bob"}]');

-- edn_q is a table function: run Datalog and get rows back.
SELECT * FROM edn_q('/tmp/demo.mentat',
  '[:find ?e ?name :where [?e :person/name ?name]]', '{}');

-- ...so it joins against native DuckDB tables.
CREATE TABLE ages(name VARCHAR, age INT);
INSERT INTO ages VALUES ('Alice', 30), ('Bob', 25);
SELECT m.name, a.age
FROM edn_q('/tmp/demo.mentat',
       '[:find ?e ?name :where [?e :person/name ?name]]', '{}') AS m(e, name)
JOIN ages a ON a.name = m.name;

-- :in inputs and time travel use the same options JSON as PostgreSQL.
SELECT * FROM edn_q('/tmp/demo.mentat',
  '[:find ?e :in [?name ...] :where [?e :person/name ?name]]',
  '{"inputs": [["Alice", "Bob"]]}');
SELECT * FROM edn_q('/tmp/demo.mentat',
  '[:find ?name :where [?e :person/name ?name]]', '{"asOf": 268435458}');

-- Pull an entity as JSON, or run a sandboxed mino script against the store.
SELECT edn_pull('/tmp/demo.mentat', '[*]', 65537);
SELECT edn_eval('/tmp/demo.mentat',
  '(mentat.store/q (mentat.store/db (mentat.store/open))
                   (quote [:find (count ?e) . :where [?e :person/name]]))');

edn_t returns a JSON tx-report. edn_q returns rows with every column as VARCHAR (cast for arithmetic, e.g. e::BIGINT); strings come back as plain text, keywords keep their leading colon. edn_pull returns JSON keyed by attribute (":person/name", plus ":db/id"). edn_eval runs in the same sandbox as PostgreSQL’s (no filesystem access, step/heap/depth limits) and is on by default (--no-default-features drops it). A DuckDB-native storage backend and typed result columns are planned; see docs/duckdb-extension-plan.md. Publishing to the DuckDB Community Extensions registry is documented in docs/registry-publishing.md.

SQLite loadable extension (`mentat_sqlite_ext`)

crates/sqlite/ext builds libmentat_sqlite.so, a SQLite loadable extension with the same four functions. Any SQLite host can load it: the sqlite3 CLI, Python’s sqlite3, or an application that calls sqlite3_load_extension. It embeds its own copy of the mentat engine (with its own SQLite) and reaches the host only through the host’s API table, so it never shares a file handle with the host. For that reason db_path must not be the host’s own database file.

cargo build --release -p mentat_sqlite_ext   # -> target/release/libmentat_sqlite.so

-- sqlite3
.load ./target/release/libmentat_sqlite
SELECT edn_t('/tmp/demo.mentat', '[{:db/ident :person/name :db/valueType :db.type/string :db/cardinality :db.cardinality/one}]');
SELECT edn_t('/tmp/demo.mentat', '[{:person/name "Alice"} {:person/name "Bob"}]');

-- edn_q returns JSON, so json_each() turns it into rows you can join.
SELECT json_extract(r.value, '$[1]') AS name
FROM json_each(edn_q('/tmp/demo.mentat',
       '[:find ?e ?name :where [?e :person/name ?name]]', '{}'), '$.results') AS r;

The functions are registered SQLITE_DIRECTONLY, so they can’t be called from views or triggers in a database schema you don’t control. Host SQLite 3.30 or newer is required. Like the DuckDB extension, it is a workspace member but not a default member. See crates/sqlite/ext/README.md.

Benchmarks

benchmarks/scale/ is a repeatable scale and load suite that runs the same scenarios against all four deployments: the embedded library, the SQLite extension, pg_mentat and the DuckDB extension. The scenarios are bulk load, point lookup, ref traversal, aggregate, predicate scan, pull, as-of/since, :in bindings, mixed read/write, a 1-128 client concurrency sweep, sustained load, and cold vs warm. Every scenario checks its answer as well as its timing. benchmarks/scale/compare.py OLD NEW flags regressions between two runs. The latest full run (r6id.metal, 1 TiB RAM, shared_buffers at 85% of RAM, up to 303M datoms) is in benchmarks/results/scale-2026-09-27T010840Z/. See benchmarks/scale/README.md to run it.

The 1.10.0 before/after runs for each fix are in benchmarks/results/{pg-autoindex,embedded-fixes,ext-cache,duckdb-quack}-*.

Indexes: automatic

Mentat manages value indexes itself, on both engines. Out of the box (mode schema), every value is indexed for lookup on PostgreSQL (an AVET index per current-state table), and on the embedded store every :db/unique attribute and :db/index ref gets a value index, created when you declare the attribute and dropped when you remove the flag. In adaptive mode mentat also watches the queries you run: an attribute that is repeatedly filtered by value (embedded) or by a range over history (PostgreSQL) gets its own partial index, and an index mentat created that goes unused for a while is dropped again. Mentat only ever drops indexes it created and recorded in its registry, never ones you made.

	Embedded (Rust / CLI)	PostgreSQL
Choose the mode	`Store::set_auto_index(AutoIndex::Adaptive)`, `MENTAT_AUTO_INDEX=adaptive`, or `.tune adaptive` in the CLI	`SET mentat.auto_index = 'adaptive'` (superuser)
See what it would do	`Store::tune_indexes(true)`, `.tune`	`SELECT * FROM mentat_tune_indexes()` (dry run)
Apply now	`Store::tune_indexes(false)`, `.tune!`	`SELECT * FROM mentat_tune_indexes(false)`
Registry	`mentat_managed_indexes` table	`mentat.managed_indexes`

Tuning also runs on its own every so often (after a number of queries on the embedded store, or of transactions on PostgreSQL). On PostgreSQL it gives up rather than wait for a lock. See docs/src/configuration.md for the thresholds and idle windows.

Scripting: mino

Every backend embeds mino, a Clojure-dialect interpreter (crates/mino), which exposes a mentat.store/* primitive surface for scripting transactions and queries. It is the SQL function edn_eval on all three: in pg_mentat behind the optional script cargo feature, and on by default in the SQLite and DuckDB extensions, where (mentat.store/open) with no argument opens the db_path you passed.

-- pg_mentat, built with --features script (off by default).
SELECT edn_eval($$
  (let [conn (mentat.store/open)]
    (mentat.store/transact conn [{:person/name "Bob"}])
    (mentat.store/q (mentat.store/db conn)
                    '[:find ?name :where [?e :person/name ?name]]))
$$);

Security — `edn_eval`

edn_eval is callable by every role by design — the extension issues no REVOKE, so PostgreSQL’s default grants EXECUTE to PUBLIC. That is safe because the interpreter is sandboxed and resource-limited:

Sandboxed. It is built with mino_rs::Interpreter::sandboxed(): the language, regex, bignum, atoms, and the in-memory mentat.store/* surface are present, but every host-filesystem primitive (slurp, spit, rm-rf, mkdir-p, file-exists?) and the file-backed store are absent (unbound). A script cannot touch the server’s filesystem.
Resource-limited. Three superuser-only (PGC_SUSET) GUCs bound each call, and an ordinary role cannot raise them for its own session:
- mentat.script_max_steps — default 10,000,000
- mentat.script_max_heap_bytes — default 64 MiB
- mentat.script_max_depth — default 2000
Exceeding any of them throws an :eval/limit error instead of hanging, exhausting memory, or overflowing the stack. statement_timeout still applies on top, via an interrupt check hook.
No privilege gain. A script runs through SPI as the calling role, so it reaches only the stores that role can already query with edn_q/edn_t. It must not be made SECURITY DEFINER — that would turn it into a privilege escalation.

The SQLite and DuckDB extensions build the same sandboxed interpreter with fixed limits (10M steps, 64 MiB heap, depth 1000). They run inside your own process, so the sandbox mainly stops a script from reaching the host filesystem; the script can read and write only the store at db_path.

License

Apache-2.0 for the workspace; the crates/mino interpreter keeps its MIT license. See LICENSE.

PGXN

PostgreSQL Extension Network

Contents

Mentat

New to EDN or Datalog?

Documentation

Embedded store (`mentat`, on SQLite)

PostgreSQL extension (`pg_mentat`)

Build and install

Use it

mentatd

DuckDB extension (`mentat_duckdb`)

Build

Use it

SQLite loadable extension (`mentat_sqlite_ext`)

Benchmarks

Indexes: automatic

Scripting: mino

Security — `edn_eval`

License

PGXN

PostgreSQL Extension Network

Contents

Mentat

New to EDN or Datalog?

Documentation

Embedded store (mentat, on SQLite)

PostgreSQL extension (pg_mentat)

Build and install

Use it

mentatd

DuckDB extension (mentat_duckdb)

Build

Use it

SQLite loadable extension (mentat_sqlite_ext)

Benchmarks

Indexes: automatic

Scripting: mino

Security — edn_eval

License

Embedded store (`mentat`, on SQLite)

PostgreSQL extension (`pg_mentat`)

DuckDB extension (`mentat_duckdb`)

SQLite loadable extension (`mentat_sqlite_ext`)

Security — `edn_eval`