See also: ROADMAP.md

v1.4.0 — PGlite WASM Extension

Release Theme This release delivers the first working PGlite extension — the moment pg_trickle’s incremental view maintenance runs in the browser. By wrapping pg_trickle_core (extracted in v0.29.0) in a thin C/FFI shim and compiling to WASM via PGlite’s Emscripten toolchain, we ship an npm package (@pgtrickle/pglite) that gives PGlite users the full DVM operator vocabulary — outer joins, window functions, subqueries, recursive CTEs — in IMMEDIATE mode. This dramatically exceeds pg_ivm’s PGlite offering (INNER joins + basic aggregates only). The release also establishes the cross-platform correctness and performance baselines that all future PGlite work builds on.

See PLAN_PGLITE.md §5 Strategy A and §7 Phase 2 for the full architecture.

PGlite WASM Build (Phase 2)

In plain terms: This takes the pg_trickle_core crate extracted in v0.29.0 and wraps it in a thin C shim that PGlite’s Emscripten-based extension build system can compile to WASM. The result is a PGlite extension package (@pgtrickle/pglite) that provides create_stream_table(), drop_stream_table(), and alter_stream_table() — all running IMMEDIATE mode inside the WASM PostgreSQL engine with the full DVM operator set.

Item Description Effort Ref
PGL-2-1 C shim for PGlite. Thin C wrapper bridging PGlite’s Emscripten environment to pg_trickle_core via Rust FFI. Handles raw_parser calls through PGlite’s built-in PostgreSQL parser. 1–2wk PLAN_PGLITE.md §5 Strategy A
PGL-2-2 DatabaseBackend for PGlite. Implement the trait for PGlite’s single-connection SPI and built-in parser. Remove advisory lock acquisition (trivial in single-connection). 3–5d PLAN_PGLITE.md §5 Strategy A
PGL-2-3 WASM bundle build. Integrate with PGlite’s extension toolchain (postgres-pglite). Produce .tar.gz WASM bundle. Target bundle size < 2 MB. 3–5d PLAN_PGLITE.md §8
PGL-2-4 TypeScript wrapper. @pgtrickle/pglite npm package with PGlite plugin API. createStreamTable(), dropStreamTable(), alterStreamTable() with full IMMEDIATE mode support. 2–3d PLAN_PGLITE.md §7 Phase 2
PGL-2-5 IMMEDIATE mode E2E tests on PGlite. Verify inner joins, outer joins, aggregates, DISTINCT, UNION ALL, window functions, subqueries, CTEs (non-recursive + recursive), LATERAL, view inlining, DISTINCT ON, GROUPING SETS. 1–2wk PLAN_PGLITE.md §4.1
PGL-2-6 PG 17 vs PG 18 parse tree compatibility. PGlite tracks PG 17; pg_trickle targets PG 18. Audit and gate any node struct differences with conditional compilation. 3–5d PLAN_PGLITE.md §8

Phase 2 subtotal: ~5–7 weeks

Correctness

ID Title Effort Priority
CORR-1 PG 17/18 parse tree node divergence audit M P0
CORR-2 Delta SQL cross-platform equivalence M P0
CORR-3 Advisory lock no-op safety proof S P1
CORR-4 IMMEDIATE trigger ordering in single-connection S P1

CORR-1 — PG 17/18 parse tree node divergence audit

In plain terms: PGlite embeds PostgreSQL 17’s parser; pg_trickle’s OpTree construction targets PostgreSQL 18 node structs. Any struct layout difference (added fields, renamed members, changed enum values) would cause the C shim to misinterpret parse trees, producing silently wrong delta SQL. Systematically diff the PG 17 and PG 18 parse tree headers (nodes/parsenodes.h, nodes/primnodes.h) and catalog every node type that pg_trickle traverses. Gate incompatible nodes behind #[cfg(pg17)] / #[cfg(pg18)] conditional compilation.

Verify: a CI job compiles pg_trickle_core against both PG 17 and PG 18 parse tree headers. A test generates OpTrees from the same SQL on both versions and asserts structural equality. Dependencies: PGL-2-6. Schema change: No.

CORR-2 — Delta SQL cross-platform equivalence

In plain terms: The same SQL view definition must produce the exact same delta SQL on native PostgreSQL 18 and PGlite (WASM + PG 17 parser). Any divergence means one platform gets wrong incremental results. Create a snapshot test suite that runs all 22 TPC-H stream table definitions through both the native and WASM DatabaseBackend implementations and asserts byte-for-byte identical delta SQL output.

Verify: snapshot comparison test passes for all 22 TPC-H queries on both platforms. Any diff is a hard failure. Dependencies: PGL-2-2, CORR-1. Schema change: No.

CORR-3 — Advisory lock no-op safety proof

In plain terms: The native extension uses pg_advisory_xact_lock() to prevent concurrent refresh of the same stream table. PGlite is single-connection — the lock acquisition is a no-op. Verify that removing the lock cannot cause re-entrancy (a trigger firing create_stream_table() from within a refresh) by auditing all SPI call paths from the PGlite DatabaseBackend for re-entrant calls.

Verify: code review + integration test that attempts re-entrant refresh from within a trigger. Must error cleanly, not corrupt state. Dependencies: PGL-2-2. Schema change: No.

CORR-4 — IMMEDIATE trigger ordering in single-connection

In plain terms: IMMEDIATE mode relies on AFTER triggers firing in a specific order when multiple source tables are modified in the same statement (e.g., a CTE with multiple INSERTs). Verify that PGlite’s trigger execution order matches native PostgreSQL’s for the trigger configurations pg_trickle creates.

Verify: integration test with multi-table CTE INSERT on PGlite; assert stream table state matches native. Dependencies: PGL-2-5. Schema change: No.

Stability

ID Title Effort Priority
STAB-1 WASM heap OOM graceful degradation M P0
STAB-2 C shim panic/unwind boundary safety S P0
STAB-3 Extension load/unload lifecycle correctness S P0
STAB-4 Native extension upgrade path (0.24 → 0.25) S P0
STAB-5 npm package version synchronization XS P1

STAB-1 — WASM heap OOM graceful degradation

In plain terms: WASM environments have a finite heap (typically 256 MB in browsers, configurable in Node). A large stream table with many operators could exhaust WASM memory during OpTree construction or delta SQL generation. The extension must detect allocation failures and return a clear PostgreSQL error rather than crashing the WASM instance (which would kill all PGlite state). Implement a memory-aware allocator wrapper or check emscripten_get_heap_size() at entry points.

Verify: stress test creating stream tables over increasingly complex views until OOM; assert PGlite remains functional and returns an actionable error. Dependencies: PGL-2-1. Schema change: No.

STAB-2 — C shim panic/unwind boundary safety

In plain terms: Rust panics must not cross the FFI boundary into C. The C shim must catch panics via std::panic::catch_unwind() and convert them to PostgreSQL ereport(ERROR) calls. Any uncaught panic in WASM would abort the entire PGlite instance. Audit every #[no_mangle] extern "C" entry point in the shim for panic safety.

Verify: test that triggers a panic path (e.g., invalid SQL) from TypeScript; assert PGlite returns a SQL error, not a WASM trap. Dependencies: PGL-2-1. Schema change: No.

STAB-3 — Extension load/unload lifecycle correctness

In plain terms: PGlite extensions can be loaded and unloaded. The C shim must free all Rust-allocated memory on unload and not leave dangling pointers or leaked state. Test the full lifecycle: load extension → create stream tables → drop stream tables → unload extension → reload extension → create new stream tables.

Verify: lifecycle test with memory profiling shows zero leaked allocations after unload/reload cycle. Dependencies: PGL-2-1, PGL-2-4. Schema change: No.

STAB-4 — Native extension upgrade path (0.27 → 0.28)

In plain terms: v0.30.0 adds PGlite support but makes no SQL-visible changes to the native extension. The upgrade migration from 0.27.0 to 0.28.0 must leave existing stream tables intact and refreshable.

Verify: upgrade E2E test confirms stream tables survive and refresh correctly after 0.27.0 -> 0.28.0.

STAB-5 — npm package version synchronization

In plain terms: The @pgtrickle/pglite npm package version must match the extension version (0.28.0). Add a CI check that verifies package.json version matches pg_trickle.control version, similar to the existing just check-version-sync target.

Verify: just check-version-sync also validates npm package version. Dependencies: PGL-2-4. Schema change: No.

Performance

ID Title Effort Priority
PERF-1 WASM vs native refresh latency benchmark M P0
PERF-2 WASM bundle size optimization (< 2 MB target) M P0
PERF-3 PGlite cold-start extension load time S P1

PERF-1 — WASM vs native refresh latency benchmark

In plain terms: WASM is expected to be 1.5–3× slower than native (per PLAN_PGLITE.md §8). Quantify the actual overhead by benchmarking IMMEDIATE-mode refresh on both platforms using the same schema + data. The overhead must stay below the threshold where IMMEDIATE mode is still faster than full re-evaluation — otherwise PGlite users would be better off just re-running the query. Establish a Criterion-like benchmark suite for PGlite (potentially using Node.js + @electric-sql/pglite).

Verify: benchmark report showing WASM refresh latency for 5 representative stream tables (scan, join, aggregate, window, recursive CTE). Document native-to-WASM overhead ratio. Dependencies: PGL-2-5. Schema change: No.

PERF-2 — WASM bundle size optimization (< 2 MB target)

In plain terms: The WASM bundle must be < 2 MB for acceptable download times in browser environments (PostGIS is 8.2 MB, pgcrypto is 1.1 MB — pg_trickle should be closer to pgcrypto). Apply wasm-opt -Oz, LTO, codegen-units = 1, strip debug info, and feature-gate large operator modules (e.g., recursive CTE, window functions) behind optional features if needed to meet the target.

Verify: CI job measures WASM bundle size after wasm-opt and fails if > 2 MB. Document size breakdown by operator module. Dependencies: PGL-2-3. Schema change: No.

PERF-3 — PGlite cold-start extension load time

In plain terms: The first CREATE EXTENSION pg_trickle in a PGlite session compiles and loads the WASM module. This must complete in < 500 ms in a browser and < 200 ms in Node.js. Measure and optimize by using streaming WASM compilation (WebAssembly.compileStreaming()) and ensuring the extension _PG_init() function does minimal work.

Verify: benchmark measuring time from CREATE EXTENSION to first create_stream_table() on fresh PGlite instance. Document cold-start time. Dependencies: PGL-2-1, PGL-2-3. Schema change: No.

Scalability

ID Title Effort Priority
SCAL-1 Stream table count ceiling in WASM S P1
SCAL-2 Wide-table OpTree memory footprint S P1
SCAL-3 Dataset size practical limit for IMMEDIATE mode S P2

SCAL-1 — Stream table count ceiling in WASM

In plain terms: Each stream table consumes memory for its OpTree, delta SQL templates, and trigger metadata. In native PostgreSQL with gigabytes of RAM this is trivial, but in a 256 MB WASM heap it matters. Determine the practical limit by creating stream tables in a loop until OOM, then document the ceiling and add a guard that errors at 80% capacity with an actionable message.

Verify: stress test documents the ceiling (e.g., “~200 stream tables with average 3-table join in 256 MB heap”). Guard errors at 80%. Dependencies: STAB-1. Schema change: No.

SCAL-2 — Wide-table OpTree memory footprint

In plain terms: A stream table over a 100-column source table produces a large OpTree and long delta SQL strings. Profile the memory consumption of OpTree construction for wide tables and ensure it fits within the WASM heap budget alongside typical stream table counts.

Verify: profile OpTree allocation for 10, 50, 100-column source tables. Document memory per stream table as a function of column count. Dependencies: PGL-2-5. Schema change: No.

SCAL-3 — Dataset size practical limit for IMMEDIATE mode

In plain terms: IMMEDIATE mode fires triggers on every DML, so overhead scales with write frequency. In a WASM environment with ~2× slower execution, determine at what dataset size (rows × columns × writes/second) IMMEDIATE mode becomes impractical. Document the breakpoint so PGlite users know when their use case has outgrown the browser and should migrate to native pg_trickle with DIFFERENTIAL mode.

Verify: benchmark with increasing write rates; document the throughput ceiling (e.g., “> 10K rows/sec INSERT rate degrades stream table latency past 100 ms”). Dependencies: PERF-1. Schema change: No.

Ease of Use

ID Title Effort Priority
UX-1 TypeScript API ergonomics and type safety S P0
UX-2 PGlite getting-started guide M P0
UX-3 WASM-context error message quality S P1
UX-4 npm package README with runnable examples S P1

UX-1 — TypeScript API ergonomics and type safety

In plain terms: The @pgtrickle/pglite TypeScript API must follow PGlite plugin conventions (PGlitePlugin interface, init() lifecycle). All methods must be fully typed — no any types. The API surface must be minimal: createStreamTable(sql), dropStreamTable(name), alterStreamTable(name, sql), listStreamTables(), and refreshStreamTable(name). Review against existing PGlite plugins (@electric-sql/pglite-repl, pglite-vector) for consistency.

Verify: TypeScript strict mode compilation with no errors. API review against PGlite plugin conventions checklist. Dependencies: PGL-2-4. Schema change: No.

UX-2 — PGlite getting-started guide

In plain terms: A docs/tutorials/PGLITE_QUICKSTART.md guide walking a user from npm install to a working React app with live stream tables in < 10 minutes. Include: install, create PGlite instance with extension, define source table + stream table, insert data, observe stream table update. Provide a CodeSandbox / StackBlitz link for zero-install try-it-now experience.

Verify: a new developer can follow the guide and see a working stream table in PGlite in a browser within 10 minutes. Dependencies: PGL-2-4, UX-1. Schema change: No.

UX-3 — WASM-context error message quality

In plain terms: Error messages from the Rust/C shim must be JavaScript-friendly: no raw pg_sys error codes, no memory addresses. Every error must include the stream table name, the failing SQL fragment, and a remediation hint. Unsupported features (DIFFERENTIAL mode, scheduled refresh, parallel workers) must error with “Not supported in PGlite:

Verify: audit all error paths in the C shim + PGlite DatabaseBackend. Every error message includes table name + remediation hint. Dependencies: PGL-2-1, PGL-2-2. Schema change: No.

UX-4 — npm package README with runnable examples

In plain terms: The npm package must have a README with: badge for PGlite compatibility, install command, 3 runnable examples (basic aggregate, join, window function), API reference, link to the full PGlite quickstart guide, and a “Limitations vs native pg_trickle” section clearly stating: no DIFFERENTIAL mode, no scheduled refresh, no parallel workers, PG 17 parser only.

Verify: README renders correctly on npmjs.com; examples are copy-pasteable into a Node.js REPL. Dependencies: PGL-2-4, UX-2. Schema change: No.

Test Coverage

ID Title Effort Priority
TEST-1 Full DVM operator E2E suite on PGlite L P0
TEST-2 PG 17/18 parse tree compatibility tests M P0
TEST-3 WASM memory stress tests M P1
TEST-4 TypeScript integration tests M P0
TEST-5 Bundle size regression gate in CI S P0

TEST-1 — Full DVM operator E2E suite on PGlite

In plain terms: Run every DVM operator (23 operators across inner join, outer join, full join, semi-join, anti-join, aggregate, distinct, union/intersect/except, subquery, scalar subquery, CTE scan, recursive CTE, lateral function, lateral subquery, window function, scan, filter, project) through IMMEDIATE mode in PGlite. This is the primary correctness gate for the WASM extension. Use a Node.js test harness with @electric-sql/pglite to run the tests headlessly.

Verify: test suite with ≥ 1 test per operator (23+ tests) passes in CI using PGlite Node.js. Test matrix: INSERT, UPDATE, DELETE for each operator. Dependencies: PGL-2-5. Schema change: No.

TEST-2 — PG 17/18 parse tree compatibility tests

In plain terms: For every parse tree node type that pg_trickle traverses, generate a test query that exercises that node, parse it on both PG 17 (PGlite) and PG 18 (native), and assert that the resulting OpTree is structurally identical. This catches version-specific divergences before they reach users.

Verify: compatibility test suite covers all node types referenced in pg_trickle_core. Any divergence is a hard failure with clear diagnostic. Dependencies: CORR-1. Schema change: No.

TEST-3 — WASM memory stress tests

In plain terms: Create increasing numbers of stream tables with increasing complexity until OOM. Verify that: (a) the guard from SCAL-1 fires at 80% capacity, (b) PGlite remains functional after the guard fires, © dropping stream tables actually frees memory. Run under different heap sizes (64 MB, 128 MB, 256 MB) to validate the guard thresholds.

Verify: stress test with 3 heap sizes completes without WASM trap. Guard fires at documented threshold. Memory reclaimed after DROP. Dependencies: STAB-1, SCAL-1. Schema change: No.

TEST-4 — TypeScript integration tests

In plain terms: Test the @pgtrickle/pglite TypeScript API end-to-end using Jest or Vitest in Node.js. Cover: create/drop/alter stream table, error handling (invalid SQL, unsupported features), plugin lifecycle (init/cleanup), and concurrent operations on different stream tables. Run as part of CI on every PR that touches pg_trickle_pglite/.

Verify: ≥ 20 TypeScript integration tests pass in CI. Test coverage report for the TypeScript wrapper shows > 90% line coverage. Dependencies: PGL-2-4, UX-1. Schema change: No.

TEST-5 — Bundle size regression gate in CI

In plain terms: Add a CI job that builds the WASM bundle, runs wasm-opt, measures the final .wasm file size, and fails if it exceeds 2 MB. Store the current size as a baseline and alert on any increase > 10%. This prevents bundle bloat as features are added.

Verify: CI job check-wasm-size runs on every PR touching pg_trickle_core/ or pg_trickle_pglite/. Fails at > 2 MB. Dependencies: PGL-2-3, PERF-2. Schema change: No.

Conflicts & Risks

  1. CORR-1 (PG 17/18 parse tree compatibility) is the highest risk. PGlite embeds PG 17; pg_trickle targets PG 18. If node struct layouts diverged significantly between versions (e.g., JoinExpr gained a field, RangeTblEntry changed a flag), the C shim must handle both layouts via conditional compilation. In the worst case, some operators may need version-specific code paths. Start this audit early — it blocks PGL-2-1 and PGL-2-2.

  2. PERF-2 (bundle size < 2 MB) may conflict with full operator coverage. If the 23-operator delta SQL generator compiles to > 2 MB, we may need to feature-gate rarely-used operators (recursive CTE, GROUPING SETS) behind cargo features. This would reduce the “full DVM vocabulary” claim and require documenting which operators are available by default. Measure early with a minimal build to establish baseline.

  3. PGlite’s Emscripten toolchain is a moving target. PGlite’s extension build system (postgres-pglite) is not yet stable. Breaking changes in the toolchain could block PGL-2-3. Pin the PGlite version and track upstream releases. Have a fallback plan: manual Emscripten compilation without the PGlite toolchain.

  4. STAB-2 (panic boundary) and STAB-1 (OOM handling) interact. A Rust OOM in WASM triggers a panic, which must not cross the FFI boundary. Both items must be implemented together: the OOM guard (STAB-1) sets a pre-panic threshold, and the catch_unwind wrapper (STAB-2) is the last-resort safety net.

  5. No prior C FFI in the codebase. The only C code is scripts/pg_stub.c (test helper). The C shim (PGL-2-1) introduces a new language and toolchain requirement. Ensure the C code is minimal (< 500 lines), well-documented, and covered by the TypeScript integration tests.

  6. TEST-1 and TEST-4 require a PGlite-based CI runner. Need Node.js 18+ with @electric-sql/pglite in CI. This is a new CI dependency. Add it to the existing CI matrix as a separate job that only runs when pg_trickle_pglite/ or pg_trickle_core/ files are modified.

v1.4.0 total: ~5–7 weeks (WASM build) + ~2–3 weeks (testing + polish)

Exit criteria: - [ ] PGL-2-1: C shim compiles and links against PGlite’s WASM PostgreSQL headers - [ ] PGL-2-2: PGlite DatabaseBackend passes all IMMEDIATE-mode operator tests - [ ] PGL-2-3: WASM bundle size < 2 MB after wasm-opt - [ ] PGL-2-4: @pgtrickle/pglite npm package published to npmjs.com - [ ] PGL-2-5: All 23 DVM operators pass E2E tests on PGlite - [ ] PGL-2-6: PG 17 parse tree differences documented and handled with #[cfg] - [ ] CORR-1: PG 17/18 parse tree audit complete; compatibility tests pass - [ ] CORR-2: Delta SQL cross-platform snapshot tests pass for all 22 TPC-H queries - [ ] CORR-3: Re-entrant refresh test passes on PGlite - [ ] CORR-4: Multi-table CTE trigger ordering matches native - [ ] STAB-1: OOM stress test: PGlite survives with actionable error - [ ] STAB-2: Panic from invalid SQL returns SQL error, not WASM trap - [ ] STAB-3: Load/unload/reload lifecycle test: zero leaked allocations - [ ] STAB-4: Extension upgrade path tested (1.3.0 → 1.4.0) - [ ] PERF-1: WASM vs native benchmark report published (≤ 3× overhead) - [ ] PERF-2: WASM bundle ≤ 2 MB (CI gated) - [ ] PERF-3: Cold-start load time < 500 ms browser, < 200 ms Node.js - [ ] TEST-1: ≥ 23 operator E2E tests pass on PGlite in CI - [ ] TEST-2: Parse tree compatibility tests cover all traversed node types - [ ] TEST-3: Memory stress tests pass under 64/128/256 MB heap sizes - [ ] TEST-4: ≥ 20 TypeScript integration tests with > 90% line coverage - [ ] TEST-5: CI check-wasm-size job passes on every PR - [ ] UX-1: TypeScript strict mode compilation: zero errors - [ ] UX-2: PGlite getting-started guide published with CodeSandbox link - [ ] UX-4: npm README renders correctly on npmjs.com - [ ] just check-version-sync passes (incl. npm package version)