Plain-language companion: v0.3.0.md

v0.3.0 — DVM Correctness, SAST & Test Coverage

Status: Released (2026-03-11).

Goal: Re-enable all 18 previously-ignored DVM correctness E2E tests by fixing HAVING, FULL OUTER JOIN, correlated EXISTS+HAVING, and correlated scalar subquery differential computation bugs. Harden the SAST toolchain with privilege-context rules and an unsafe-block baseline. Expand TPC-H coverage with rollback, mode-comparison, single-row, and DAG tests.

DVM Correctness Fixes

In plain terms: The Differential View Maintenance engine — the core algorithm that computes what changed incrementally — had four correctness bugs in specific SQL patterns. Queries using these patterns were silently producing wrong results and had their tests marked “ignored”. This release fixes all four: HAVING clauses on aggregates, FULL OUTER JOINs, correlated EXISTS subqueries combined with HAVING, and correlated scalar subqueries in SELECT lists. All 18 previously-ignored E2E tests now pass.

Item Description Status
DC1 HAVING clause differential correctness — fix COUNT(*) rewrite and threshold-crossing upward rescan (5 tests un-ignored) ✅ Done
DC2 FULL OUTER JOIN differential correctness — fix row-id mismatch, compound GROUP BY expressions, SUM NULL semantics, and rescan CTE SELECT list (5 tests un-ignored) ✅ Done
DC3 Correlated EXISTS with HAVING differential correctness — fix EXISTS sublink parser discarding GROUP BY/HAVING, row-id mismatch for Project(SemiJoin), and diff_project row-id recomputation (1 test un-ignored) ✅ Done
DC4 Correlated scalar subquery differential correctness — rewrite_correlated_scalar_in_select rewrites correlated scalar subqueries to LEFT JOINs before DVM parsing (2 tests un-ignored) ✅ Done

DVM correctness subtotal: 18 previously-ignored E2E tests re-enabled (0 remaining)

SAST Program (Phases 1–3)

In plain terms: Adds formal static security analysis (SAST) to every build. CodeQL and Semgrep scan for known vulnerability patterns — for example, using SECURITY DEFINER functions without locking down search_path, or calling SET ROLE in ways that could be abused. Separately, every Rust unsafe {} block is inventoried and counted; any PR that adds new unsafe blocks beyond the committed baseline fails CI automatically.

Item Description Status
S1 CodeQL + cargo deny + initial Semgrep baseline — zero findings across 115 Rust source files ✅ Done
S2 Narrow rust.panic-in-sql-path scope — exclude src/dvm/** and src/bin/** to eliminate 351 false-positive alerts ✅ Done
S3 sql.row-security.disabled Semgrep rule — flag SET LOCAL row_security = off ✅ Done
S4 sql.set-role.present Semgrep rule — flag SET ROLE / RESET ROLE patterns ✅ Done
S5 Updated sql.security-definer.present message to require explicit SET search_path ✅ Done
S6 scripts/unsafe_inventory.sh + .unsafe-baseline — per-file unsafe { counter with committed baseline (1309 blocks across 6 files) ✅ Done
S7 .github/workflows/unsafe-inventory.yml — advisory CI workflow; fails if any file exceeds its baseline ✅ Done
S8 Remove pull_request trigger from CodeQL + Semgrep workflows (no inline PR annotations; runs on push-to-main + weekly schedule) ✅ Done

SAST subtotal: Phases 1–3 complete; Phase 4 rule promotion tracked as post-v0.3.0 cleanup

TPC-H Test Suite Enhancements (T1–T6)

In plain terms: TPC-H is an industry-standard analytical query benchmark — 22 queries against a simulated supply-chain database. This extends the pg_trickle TPC-H test suite to verify four additional scenarios that the basic correctness checks didn’t cover: that ROLLBACK atomically undoes an IVM stream table update; that DIFFERENTIAL and IMMEDIATE mode produce identical answers for the same data; that single-row mutations work correctly (not just bulk changes); and that multi-level stream table DAGs refresh in the correct topological order.

Item Description Status
T1 __pgt_count < 0 guard in assert_tpch_invariant — over-retraction detector, applies to all existing TPC-H tests ✅ Done
T2 Skip-set regression guard in DIFFERENTIAL + IMMEDIATE tests — any newly skipped query not in the allowlist fails CI ✅ Done
T3 test_tpch_immediate_rollback — verify ROLLBACK restores IVM stream table atomically across RF mutations ✅ Done
T4 test_tpch_differential_vs_immediate — side-by-side comparison: both incremental modes produce identical results after shared mutations ✅ Done
T5 test_tpch_single_row_mutations + SQL fixtures — single-row INSERT/UPDATE/DELETE IVM trigger paths on Q01/Q06/Q03 ✅ Done
T6a test_tpch_dag_chain — two-level DAG (Q01 → filtered projection), refreshed in topological order ✅ Done
T6b test_tpch_dag_multi_parent — multi-parent fan-in (Q01 + Q06 → UNION ALL), DIFFERENTIAL mode ✅ Done

TPC-H subtotal: T1–T6 complete; 22/22 TPC-H queries passing

Exit criteria: - [x] All 18 previously-ignored DVM correctness E2E tests re-enabled - [x] SAST Phases 1–3 deployed; unsafe baseline committed; CodeQL zero findings - [x] TPC-H T1–T6 implemented; rollback, differential-vs-immediate, single-row, and DAG tests pass - [x] Extension upgrade path tested (0.2.3 → 0.3.0)