Upgrading pg_turbovec

pg_turbovec follows SemVer with a strict data-format compatibility contract:

  • Patch releases (X.Y.Z → X.Y.Z+1) never change the on-disk index format. Drop in the new shared library, restart, scan; no REINDEX required.
  • Minor releases (X.Y → X.Y+1) may bump the on-disk format version when there’s a clear performance / correctness win that can’t be expressed at the existing version. When they do, the new binary detects the old format on first open and emits a REINDEX-pointed ERROR rather than silently corrupting or returning bad results.
  • Major releases (X → X+1) may make breaking changes to the SQL surface in addition to the on-disk format.

Every release that bumps the on-disk format ships with:

  • A clear ERROR from ambeginscan saying “this index was built under pg_turbovec ≤ X.Y; run REINDEX INDEX <name>; to migrate”.
  • A row in the migration matrix below.
  • A test under src/lib.rs (search for legacy_v1-style names) that exercises the detection primitive.

Migration matrix

From To Required action Notes
1.0.x (side-table only) 1.3.0+ REINDEX INDEX <name>; per index The 1.0.x indexes have an empty main fork and a turbovec.am_storage row; v1.3.0 drops that table during ALTER EXTENSION pg_turbovec UPDATE (migrations/005_pg_turbovec_v1.3.0.sql). The index is unscannable until reindexed.
1.1.x (side-table only) 1.3.0+ REINDEX INDEX <name>; Same as 1.0.x.
1.2.x with --features relfile_storage 1.3.0+ REINDEX INDEX <name>; The 1.2 relfile preview wrote MetaPageData::version = 1. v1.3.0 introduced the pre-baked SIMD-blocked layout + codebook, bumping VERSION to 2. The detection primitive in src/index/page.rs::MetaPageData::is_legacy_v1() fires on these.
1.2.x without relfile_storage 1.3.0+ REINDEX INDEX <name>; Same boat as 1.0/1.1.
1.3.x 1.4.0+ REINDEX INDEX <name>; per index v1.4.0 introduced the persisted rotation matrix in the relfile (MetaPageData::version 2→3). v1.3.x indexes have an empty rotation chain so the new binary detects them via MetaPageData::is_legacy_v2() and ERRORs out cleanly. The lazy QR was the warm-scan hotspot (~64.8% self time on dbpedia-1M), so persisting the matrix closes the gap to pgvector HNSW.
1.3.x → 1.3.x+1 (patch) none none Wire format is frozen across patch releases.
1.4.x → 1.4.x+1 (patch) none none Wire format is frozen across patch releases.
1.4.x 1.5.0+ none v1.5.0 (Phase R-3) is a scan-side change only: the ambeginscan cache-fill path now mmaps the deterministic static regions of the relfile (blocked codes + rotation matrix + inline codebook) instead of pulling them through the buffer manager. The on-disk format (MetaPageData::version = 3) is byte-identical to v1.4.x. No REINDEX. The fall-back GUC turbovec.mmap_static_blocked = off reverts to the v1.4.x scan path on a per-session basis. See docs/ARCHITECTURE.md § “Index AM · mmap isolation contract” for the consistency story.
1.5.x → 1.5.x+1 (patch) none none Wire format is frozen across patch releases.
1.5.x 1.6.0+ none v1.6.0 (Phase W) is a build-side change only: ambuild now streams the heap scan into IdMapIndex::add_with_ids in chunks bounded by maintenance_work_mem (capped at 1 GiB) instead of accumulating the entire heap-scan output in a single Vec<f32>. Peak CREATE INDEX memory drops from ~121 GiB to ~16 GiB at 10 M × 1536-d × 4-bit. The on-disk format (MetaPageData::version = 3) is byte-identical to v1.5.x. No REINDEX required. Existing v1.5.x indexes continue to work unchanged; the v1.6.0 binary’s ambuild path simply uses less memory on the next CREATE INDEX / REINDEX.
1.6.x → 1.6.x+1 (patch) none none Wire format is frozen across patch releases.
1.6.x 1.7.0+ none v1.7.0 (Phase W-2) is a build-side change only: ambuild now writes packed_codes to relfile pages, materialises the SIMD-blocked layout via prepare_eager(), drops packed_codes via the new IdMapIndex::take_packed_codes() (turbovec 0.7.0), then writes the blocked + rotation chains and stamps the meta page LAST. Peak CREATE INDEX memory drops from ~22.5 GiB to ~15 GiB at 10 M × 1536-d × 4-bit (8× total reduction vs pre-Phase-W). The on-disk format (MetaPageData::version = 3) is byte-identical to v1.6.x. No REINDEX required. Existing v1.6.x indexes continue to work unchanged; the v1.7.0 binary’s ambuild path simply uses less memory on the next CREATE INDEX / REINDEX.
1.7.x → 1.7.x+1 (patch) none none Wire format is frozen across patch releases. v1.7.1 specifically reverts the v1.7.0 (Phase W-2) build-path reordering after the 10 M × 1536-d validation on meh showed the split-write design made the build 53% slower (5052 → 7748 s) and used 2.7 GiB of swap (vs 0 in v1.6.0) without actually lowering peak RSS — the pinned-shared-buffer component of ps -o rss ate the predicted savings. v1.7.2 is a test-only patch that adds automated #[pg_test]s for the upgrade matrix (Phase Y): forged-meta-page detection of pre-v1.4 wire formats and migration-file drift checks. v1.7.3 upgrades the turbovec kernel fork from the v0.7.0-era 6e80a59 to a fork rebased onto upstream v0.9.0 (d3d468e), fixing a kernel bug where x86_64 CPUs WITHOUT AVX2 returned silently-wrong / repeated top-k from indexed ANN scans (the perm0-interleave scalar-fallback bug, upstream PR #108 / issue #106). TQ+ calibration is adopted as identity (no recall or wire change); security hardening (MAX_DIM, NaN/Inf rejection) comes along. The on-disk format is byte-identical across v1.6.0 / v1.7.0 / v1.7.1 / v1.7.2 / v1.7.3; no REINDEX needed when upgrading or downgrading among them. Pre-AVX2 x86_64 users specifically should take v1.7.3 to clear the wrong-results bug. § “Phase W-2 reverted in v1.7.1” and docs/PRODUCTION.md § “Known issues”.
1.7.x 1.8.0+ none v1.8.0 is a competitive-parity minor: iterative index scan (turbovec.iterative_scan, turbovec.max_scan_tuples), parallel index build (turbovec.build_parallelism), cold-scan latency cut (lazy id_to_slot on the read path), and additive || concat + halfvec +/-/* arithmetic. All changes are scan-side, build-side, or additive SQL surface; the on-disk relfile format is byte-identical to v1.7.x (MetaPageData::version = 3). No REINDEX needed. The new GUCs default to pgvector-equivalent behaviour (iterative_scan = relaxed_order). The additive operators/functions are created by the generated 1.7.3--1.8.0 upgrade script; ALTER EXTENSION pg_turbovec UPDATE TO '1.8.0'; is sufficient..
1.8.x → 1.8.x+1 (patch) none none Wire format is frozen across patch releases.
1.8.x 1.9.0+ none v1.9.0 adds turbovec.oversample (tunable recall — fetch search_k * oversample quantized candidates, reorder-queue trims to exact top-k), plus test-coverage hardening and the first published benchmark. The GUC is additive and defaults to 1.0 (no-op). On-disk format byte-identical to v1.7.x / v1.8.x (MetaPageData::version = 3). No REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.9.0'; is sufficient.
1.9.x → 1.9.x+1 (patch) none none Wire format is frozen across patch releases. v1.9.1 is bench-results-only (the AVX2 latency-frontier run on arnold + the honest positioning correction it produced); no source or wire change.
1.4.x – 1.9.x (flat) 1.10.0+ none v1.10.0 adds the IVF coarse-quantizer layer and bumps MetaPageData::version 3 → 4 — BUT a v1.10.0 binary reads any existing v3 (flat) index as a lists = 0 flat index, byte-compatible. No REINDEX needed to upgrade. ALTER EXTENSION pg_turbovec UPDATE TO '1.10.0'; registers the new lists / assign_dups reloptions and turbovec.probes / turbovec.max_probes GUCs. Opt into IVF only by rebuilding a chosen index with WITH (lists = N) (recommended N ≈ sqrt(n)); that index is then v4-IVF, while un-rebuilt indexes stay v4-flat..
1.10.x → 1.10.x+1 (patch) none none Wire format is frozen across patch releases. v1.10.1 is bench-results-only (the AVX2 IVF warm-p50 measurement confirming the ~5×-vs-full-scan latency win); no source or wire change.
1.10.x 1.11.0+ none v1.11.0 hardens IVF for production: it survives VACUUM via tombstones (a v4-ADDITIVE per-slot bitmap chain) instead of silently degrading to a flat scan, and adds the turbovec.index_is_degraded(regclass) function + a throttled degradation WARNING; IVF k-means builds ~7.8× faster (build-internal). Wire stays MetaPageData::version = 4 — the tombstone bitmap + ivf_degraded flag are additive, so pre-1.11.0 v4 indexes read as not-degraded/no-tombstones. No REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.11.0'; registers the new function.
1.11.x → 1.11.x+1 (patch) none none Wire format is frozen across patch releases. v1.11.1 is bench-results-only (the Phase A-2 IVF latency frontier measurement + Phase B out-of-core design); no source or wire change.
1.11.x 1.12.0+ none v1.12.0 makes the IVF build out-of-core (spills the corpus to a PG temp file; peak RSS at 1M×1024-d ~14 GiB → ~7.1 GiB, 5M buildable on a 31 GiB host). Build-internal only — the on-disk relfile is byte-identical to a v1.11.x in-memory build for the same input, MetaPageData::version stays 4. No REINDEX; the benefit applies to the next CREATE INDEX / REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.12.0'; is sufficient.
1.12.x → 1.12.x+1 (patch) none none Wire format is frozen across patch releases.
1.12.x 1.13.0+ none v1.13.0 adds out-of-core IVF query (turbovec.out_of_core enum, default auto): an IVF index larger than RAM can now be served cell-scoped (caches only bounded metadata + an mmap; copies just the probed cells' code ranges per query). Scan-path only — results are identical to the whole-load path, MetaPageData::version stays 4. No REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.13.0'; is sufficient.
1.13.x → 1.13.x+1 (patch) none none Wire format is frozen across patch releases. v1.13.1 is docs + bench only (Phase C metadata-filtering guide + allowlist crossover measurement + drift fixes); no source-logic or wire change.
1.13.x 1.14.0+ none v1.14.0 (Phase D) adds the multivector / hybrid SQL surface: turbovec.max_sim / max_sim_cosine (ColBERT-style MaxSim re-rank over vector[]) and turbovec.rrf_score (reciprocal rank fusion). Additive functions only — no index-AM change, MetaPageData::version stays 4. No REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.14.0'; is sufficient.
1.14.x → 1.14.x+1 (patch) none none Wire format is frozen across patch releases.
1.14.x 1.15.0+ none v1.15.0 (Phase C follow-up) adds the operator-path allowlist: a session GUC turbovec.allowlist (CSV of heap TIDs) that flows a pre-materialized id-set into the ORDER BY emb <=> q scan for in-kernel block-skip pushdown on flat + IVF, plus the turbovec.tid_to_bigint(tid) encoder. Additive GUC + function only — no index-AM scan-key rewrite, MetaPageData::version stays 4. No REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.15.0'; is sufficient.
1.15.x → 1.15.x+1 (patch) none none Wire format is frozen across patch releases. v1.15.1 is a build-only fix (the Phase B-4 BufFile spill didn’t compile on pg13/14/15/18 from v1.12.0–v1.15.0); no wire or runtime change on pg16/pg17.
1.15.x 1.16.0+ none v1.16.0 (Phase F-1) adds turbovec.colbert_search — index-accelerated stage-1 ColBERT late interaction (backend-cached token index + exact max_sim stage-2 rerank). Additive function only; the token index is backend-cache-only (no relfile), MetaPageData::version stays 4. No REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.16.0'; is sufficient.
1.16.x → 1.16.x+1 (patch) none none Wire format is frozen across patch releases.
1.17.x → 1.17.x+1 (patch) none none Wire format is frozen across patch releases. v1.17.1 is docs + bench only (the F-2 ColBERT recall win confirmed cross-domain on NFCorpus); no source or wire change (single-vector stays v4, colbert v5).
1.17.x 1.18.0+ none v1.18.0 (Tier-1 IVF latency) lowers the default turbovec.search_k 100 → 32 (the recheck floor, not the scan, dominates per-query latency; recall@10 plateaus by ~25) and documents that raising WITH (assign_dups = M) reaches matched recall at fewer probes. Scan-path / default-tuning only — no SQL surface or wire change, MetaPageData::version stays 5. No REINDEX (the assign_dups lever is opt-in and only takes effect on a fresh build). ALTER EXTENSION pg_turbovec UPDATE TO '1.18.0'; is sufficient.
1.18.x 1.19.0+ none v1.19.0 removes the direct relfile mmap: all index data is now read through PostgreSQL’s buffer manager (ReadBufferExtended). Required for managed/sandboxed Postgres. Read-path only — no SQL surface or wire change, MetaPageData::version unchanged; out-of-core IVF serving is preserved (the cell-scoped gather reads only probed cells' pages via the buffer manager). turbovec.mmap_static_blocked is deprecated to a no-op. No REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.19.0'; is sufficient; size shared_buffers to hold the hot index for best cold-fill latency.
1.19.x 1.20.0+ none v1.20.0 lands IVF scaling: a sublinear two-level coarse quantizer (O(lists)→O(√lists) cell selection, computed in-memory from the persisted centroids, auto for lists>4096), a turbovec.scan_parallelism GUC (parallel per-query fine-scan, opt-in), and a memory-bounded byte-identical parallel build. In-memory / scan-path / build-path only — no wire change (MetaPageData::version stays 5), one new GUC. No REINDEX — existing v4/v5 IVF indexes get the sublinear coarse + parallel scan on the next scan with no rebuild. ALTER EXTENSION pg_turbovec UPDATE TO '1.20.0'; is sufficient.
1.20.0 1.20.1+ none v1.20.1 is a CRITICAL PERF FIX, not a feature change. turbovec.iterative_scan default flips relaxed_order → off. Root cause: PostgreSQL’s reorder queue (IndexNextWithReorder) can only return a tuple early when the AM’s advertised ORDER BY value is exact; pg_turbovec always advertises NEG_INFINITY (opclass-agnostic safety), so under the old default the executor was forced to drive the AM’s own iterative-refill schedule to completion on EVERY ORDER BY ... LIMIT query — measured ~450x slower (SIFT-1M/128d: ~2ms vs ~900ms) than with off. Scan-side GUC-default-only — no wire change, no SQL surface change, MetaPageData::version stays 5. No REINDEX. If your workload relies on relaxed_order’s under-return-avoidance for a selective WHERE filter, opt back in with SET turbovec.iterative_scan = relaxed_order;. ALTER EXTENSION pg_turbovec UPDATE TO '1.20.1'; is sufficient.
1.20.x 1.21.0+ none v1.21.0 (Phase G-1, ) adds a small in-memory undirected graph (Vamana/HNSW-lite, fixed out-degree 16, symmetrized for recall-safe greedy search) over the IVF coarse centroids: coarse_probe on the out-of-core cell-scoped path can navigate the graph instead of scoring every centroid once lists >= 4096. The graph is built ONCE PER BACKEND, IN-MEMORY, from the already-persisted coarse centroids (deterministic; see centroid_graph_build_deterministic / ivf_coarse_graph_build_is_deterministic_across_cache_rebuilds) — nothing new is persisted, MetaPageData::version stays 5. One new GUC, turbovec.coarse_graph (off/auto/on, default auto). No REINDEX — existing v4/v5 IVF indexes get the graph (when lists >= 4096) on the next scan with no rebuild. ALTER EXTENSION pg_turbovec UPDATE TO '1.21.0'; is sufficient. Note: the v1.20.0 row above (and its CHANGELOG entry) describes a “sublinear two-level coarse quantizer” that was never actually implemented in that release — v1.20.0 shipped parallel k-means seeding/build and turbovec.scan_parallelism only; coarse_probe stayed the plain O(lists·dim) linear scan until this release’s graph. v1.21.0 is the first release to actually ship sublinear coarse-cell selection.
1.21.x 1.22.0+ none v1.22.0 is a repo-cleanup release, no functional change. turbovec.mmap_static_blocked — a deprecated no-op since v1.19.0 (it toggled a relfile-mmap fast path deleted that release) — is removed after a three-minor deprecation window (v1.19.0 warn → v1.20.0/v1.21.0 still-warning → v1.22.0 remove), per AGENTS.md’s SQL-surface-removal policy. SET turbovec.mmap_static_blocked = ... now errors like any unknown GUC instead of silently no-op'ing. Also: fixed a stale dead-code warning, deleted a test made meaningless by the mmap removal, cargo fmt’d the whole tree (244 pre-existing formatting violations, purely cosmetic — fmt-check was never wired into real CI before this release, only into an already-dead .woodpecker/ci.yaml, which is also removed), and fixed literal \uXXXX escape-sequence artifacts in several doc files. No wire-format change (MetaPageData::version stays 5), no other SQL surface change. No REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.22.0'; is sufficient.
1.22.0 1.22.1+ none v1.22.1 closes a real fraction of the IVF build-cliff gap: gemm_lloyd_assign’s Lloyd-loop cross-term GEMM (dominant k-means training cost at high lists) now runs Parallelism::Rayon(0) instead of Parallelism::None, respecting turbovec.build_parallelism automatically. Bit-identical centroid output confirmed (empirically and via a new regression test). Measured on real GIST-1M-scale k-means training (16-core AVX-512 a cloud VM): 2686.6s → 768.4s, a 3.50× speedup. Scan/build-path only — no wire-format change, no SQL surface change, no new/changed GUC or reloption. No REINDEX: this changes build wall clock only, not the on-disk bytes. ALTER EXTENSION pg_turbovec UPDATE TO '1.22.1'; is sufficient.
1.22.1 1.22.2+ none v1.22.2 raises turbovec.probes’s default from 8 to 16 — the old default capped out-of-the-box recall at R@10=0.796 (SIFT-1M) / R@10=0.407 (GIST-1M), well below any reasonable SLO. probes=16 measures R@10=0.918 / 0.557 respectively for ~1.5-1.6× the latency. Existing sessions/deployments that explicitly SET turbovec.probes are unaffected. Scan-side default only — no wire-format change, no SQL surface change. No REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.22.2'; is sufficient.
1.25.1 1.26.0+ none v1.26.0 (Phase G-2d(a)) adds a partitioned/merge PARALLEL build for the graph index kind (WITH (graph = true)) so it scales past the single-pass serial ceiling (which didn’t complete at 5M rows). Partition into P shards → build each in parallel → stitch via a parallel cross-shard refinement + reverse-edge pass. New GUC turbovec.graph_build_partitions (int, default auto: derives P from corpus size + build-pool budget; 0/1 forces single-pass; N forces N shards). Build-time change only — emits the IDENTICAL on-disk v6 CSR shape, so no wire-format change, no new operators/types/functions, and no REINDEX (existing graph indexes unaffected; only NEW graph builds use the parallel path). Verified: recall parity (partitioned matches or BEATS single-pass, 0.958→0.996 R@10 in a findable regime), ~8× build speedup (P=16, 200k rows, 8-core box), bit-identical determinism across (corpus, seed, P) and rayon pool sizes. ALTER EXTENSION pg_turbovec UPDATE TO '1.26.0'; is sufficient.
1.27.0 1.27.1+ none v1.27.1 (Phase Q-4a) parallelizes the IVF k-means build — a build-SPEED change only. The persisted IVF centroids + assignment are BYTE-IDENTICAL to v1.27.0 for a fixed (corpus, seed, lists, dim); no wire change (stays v7), no SQL-surface change, no REINDEX. Two remaining serial hot loops (gemm_lloyd_assign’s per-row argmin, rotate_corpus_into’s GEMM) were parallelized bit-identically; measured ~1.91× faster IVF builds (sub-linear — Lloyd iterations are sequentially dependent). Only NEW WITH (lists = N) builds are affected (faster); existing indexes unchanged. ALTER EXTENSION pg_turbovec UPDATE TO '1.27.1'; is a no-op upgrade.
1.27.1 1.27.2+ none v1.27.2 (Phase Q-4b) defers the per-row O(dim²) rotation in IVF k-means reservoir sampling to only the rows kept in the reservoir (≤ 256·lists) instead of all N accepted rows — a build-SPEED change only. The persisted IVF centroids + codes are BYTE-IDENTICAL to v1.27.1 for a fixed (corpus, seed, lists, dim); no wire change (stays v7), no SQL-surface change, no REINDEX. Only NEW WITH (lists = N) builds are affected (faster). ALTER EXTENSION pg_turbovec UPDATE TO '1.27.2'; is a no-op upgrade.
1.27.2 1.27.3+ none v1.27.3 (Phase Q-4c) batches the IVF k-means reservoir rotation into one parallel BLAS GEMM instead of a scalar per-row rotation — a build-SPEED change only (~6.3× faster 1M builds; cleared the 10M build cliff, 26 min vs prior DNF). Persisted centroids + codes are BYTE-IDENTICAL to v1.27.2 for a fixed (corpus, seed, lists, dim); no wire change (stays v7), no SQL change, no REINDEX. Only NEW WITH (lists = N) builds are affected (faster). ALTER EXTENSION pg_turbovec UPDATE TO '1.27.3'; is a no-op upgrade.
1.27.3 1.28.0+ none v1.28.0 upgrades the pgrx framework 0.17→0.19.1 and adds PostgreSQL 19 (beta1) support. No wire change (stays v7), no SQL-surface change, no REINDEX — ALTER EXTENSION pg_turbovec UPDATE TO '1.28.0'; suffices. Build-from-source floor rises to Rust 1.96 / edition 2024 / cargo-pgrx 0.19.1. PG19 support is experimental until PG19 RC/GA.
1.28.0 1.28.1+ none v1.28.1 is packaging-only: adds the Nix flake (nix build github:gburd/pg_turbovec#pg_turbovec_NN, NN in 13–19). No code/wire/SQL change, no REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.28.1'; is a no-op upgrade.
1.28.1 1.28.2+ none (but REINDEX a corrupt index) v1.28.2 detects a duplicate-id corrupt .tvim relfile (which pg_resetwal/unclean shutdown can leave) at read/open time and ERRORs with HINT: REINDEX INDEX <name>; instead of silently mis-serving reads while failing every write. No wire/SQL change; ALTER EXTENSION pg_turbovec UPDATE TO '1.28.2'; suffices. If an index is ALREADY corrupt, REINDEX INDEX <name>; (or ... CONCURRENTLY) rebuilds a clean id bijection from the heap.
1.28.2 1.28.3+ none v1.28.3 (managed-PG readiness) adds CHECK_FOR_INTERRUPTS at lock-free scan/build/write boundaries (statement_timeout / cancel now take effect promptly), drift-check gates that keep durability 100% GenericXLog and all GUCs per-session, and docs/DEPLOYING_ON_MANAGED_POSTGRES.md. No wire/SQL change; ALTER EXTENSION pg_turbovec UPDATE TO '1.28.3'; suffices.
1.28.3 1.28.4 none (but recover a corrupt index) v1.28.4 (corruption fix) makes idx.slot_to_id().len() the SINGLE source of truth for the persisted row count — the deferred aminsert flush no longer passes a separately-incremented PersistState.n_vectors counter that could drift and leave a meta page over-reading the ids chain into zeroed “id 0” slots (the reported “duplicate ids in .tvim (id 0 in more than one slot)” corruption). A hard runtime guard aborts rather than persisting a mismatched meta page. Adds read-only turbovec.turbovec_check(regclass) (additive) so the corruption is detectable WITHOUT attempting a write. With the write-path source fixed, REINDEX now durably repairs. No wire change (stays v7), no REINDEX required by the upgrade; ALTER EXTENSION pg_turbovec UPDATE TO '1.28.4'; suffices. If an index is ALREADY corrupt, REINDEX INDEX <name>; (now durable) or DROP + CREATE recovers it — the fix stops NEW corruption, it does not repair a pre-existing one. Known gap ©: the .tvim id table is still not fully WAL-crash-safe; pg_resetwal/unclean shutdown can re-create the corruption, which the insert/read paths ERROR on loudly (v1.28.2) and turbovec_check now surfaces.
1.28.4 1.29.0+ none v1.29.0 adds the partitioned-scale cookbook (docs/PARTITIONED_SCALE.md: scale to 1-10B vectors today via hash-partitioning + per-partition turbovec indexes + PG native Merge Append, no AM code) and the 1-bit sign-BQ foundation (WITH (bit_width = 1) reloption accepted; the build path is not yet implemented and ERRORs clearly). Partition pruning for very large N (Phase S-1) is designed but deferred. Additive only, no wire change (stays v7), no REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.29.0'; suffices.
1.29.0 1.29.1+ none v1.29.1 ships the in-place ALTER EXTENSION UPDATE scripts that v1.28.4 was missing (turbovec_check() is now created on in-place upgrade, not just fresh install). Packaging only, no wire/code change, no REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.29.1'; suffices.
2.8.2 2.8.3 none (no REINDEX) bit_width = 4 + IVF measured at 1M — flat wins at every target. Binary byte-identical to 2.8.2. Answers the production user’s question with data: flat is 6.08 ms at R@10 = 1.000 (window 32), while lists = 1024 is 62 % slower where it works and cannot reach R@10 ≥ 0.98 at any probes. The conclusion needs no timing at all — at probes = 128, widening the rerank window 32 → 2000 leaves recall at exactly 0.959 across all 8 windows, and recall is CPU-independent. Also retires the OOM worry for this configuration: peak build RSS is 2.6 GiB at 1M/1024-d with maintenance_work_mem = '4GB' (real peaks, 0.25 s sampling; IVF’s peak below flat’s) — the 20.3 GB kill came from leaving it unbounded. Real cost is the 6.8× build time. Caveats recorded: all latency rows contention-flagged with the filter unavailable (bias runs against IVF), a second unexplained corpus-identity discrepancy versus § 0.6e, and no answer for a ≲ 0.90 target above 1M. ALTER EXTENSION pg_turbovec UPDATE TO '2.8.3';
2.10.3 2.11.0 none required; REINDEX 2/¾-bit flat/IVF indexes that took writes from long-lived connections during VACUUM (restart to load the library) MINOR — adopt upstream turbovec 1.1.1 (staged 2/4-bit search); fix silent index-entry corruption (since v1.29.1: a long-lived writer re-spliced stale rows every commit, so after VACUUM + TID reuse a live row could carry another row’s codes or a vacuumed entry could come back; turbovec_check stayed clean; REINDEX clears existing damage) and a per-scan memory leak (since v1.8.0: a long-lived backend scanning an index under concurrent writes grew ~one in-memory index per observed commit until OOM-killed; recommended upgrade for pooled-connection deployments). On aarch64 (dotprod) and x86 AVX-512 VBMI+VNNI hosts, indexes of ≥ 32,768 rows search in stages (sign-plane shortlist → lower-plane ranking → exact rescore). Measured on Graviton4, 1M × 1024-d real Cohere, flat, warm end-to-end p50: 4-bit 1.14–1.17× faster, 2-bit 1.05–1.12×, recall@10 unchanged; the turbovec kernel alone is 3–6.5× faster. Cold-backend p50 4-bit 514 → 477 ms (new fork carry #4 parallelizes the planes cold-open repack, which stock 1.1.1 does serially at 279 ms / 1.09 s). AVX2-only x86 is unchanged. No wire change (stays v8): a 1M-row index built by 2.10.3 and 2.11.0 is byte-identical (sha256). Minor because the candidate SET can differ slightly from 2.10.3 (scores exact; top-10 id set identical for 98.5–100% of queries); TURBOVEC_4BIT_PLANES=0 / TURBOVEC_2BIT_PLANES=0 in the postmaster environment restores the whole-index scan. Evidence: benches/results/tv111_arm_20261005/. ALTER EXTENSION pg_turbovec UPDATE TO '2.11.0'; + restart.
2.10.2 2.10.3 none (no REINDEX) Cold-scan latency cut ~3×. Every cold backend rebuilds the SIMD-blocked code layout from the row-major packed codes at index-open (v7+ persists only the codes). That repack was single-threaded and dominated cold latency; it now runs in PARALLEL across block-aligned ranges. Measured A/B on identical hardware/corpus/index (c7i.4xlarge, 16 vCPU, AVX-512; 1M × 1024-d 4-bit flat): cold-backend p50 1766 ms → 566 ms (3.1×), warm p50 unchanged (30.4 → 30.6 ms — a warm backend never repays the repack). The parallel repack (turbovec fork carry #3, rev 47a26a3) is byte-identical to serial, pinned by turbovec’s parallel_repack_is_byte_identical_to_serial. Also RETRACTS the old “we LOSE ~490×” latency scoreboard — a corrected end-to-end benchmark (literal query vectors, top-level Execution Time) shows flat-bw4 at 5.2 ms / R@10 1.000 BEATS HNSW at R@10 ≥ 0.95. No wire change (stays v8), index bytes unchanged. ALTER EXTENSION pg_turbovec UPDATE TO '2.10.3';
2.10.1 2.10.2 none (no REINDEX) Discoverability + documentation: answering “you don’t support ANN”. lists defaults to 0, so a plain CREATE INDEX ... USING turbovec builds a FLAT exact scan and nothing advertised the IVF layer – an evaluator concluded from that experience that pg_turbovec lacks ANN and chose another extension. A flat build over 100k rows now emits a NOTICE naming WITH (lists = N) (a NOTICE, not a WARNING: flat is often the better choice, so it must not read as a fault). New README sections answer four common objections with measurements – including the two where the objection is CORRECT (HNSW is genuinely faster on latency; our own build memory was genuinely 121 GiB vs HNSW’s 16.9 GiB before v2.10.1 fixed it) – plus a full lists tuning guide. The default stays lists = 0, deliberately: changing it to sqrt(n) was rejected on our own data, which shows flat at 6.08 ms / recall 1.000 versus lists = 1024 at 62 % slower and capped at 0.959 for the default bit_width = 4. No wire change (stays v8), index bytes unchanged. ALTER EXTENSION pg_turbovec UPDATE TO '2.10.2';
2.10.0 2.10.1 none (no REINDEX) Build-memory fix: CREATE INDEX peak cut ~3.5x. The build callback had no memory-context management, and Vector is stored as CBOR – so FromDatum palloc’d a decoded buffer per row and all of them accumulated in the long-lived ambuild context. The spill streamed the corpus to disk while PostgreSQL held a decoded copy of the whole thing in RAM. Now uses a per-tuple context reset after every row (with CorpusSpill::new_in keeping the lazily-opened BufFile in the long-lived context). Measured 2M x 1024-d: peak 12.16 -> 3.45 GiB, 16 % faster. Also bounds the Lloyd cross matrix, which was quadratic in lists (9.54 GiB at lists = 3162), and corrects turbovec.build_parallelism’s description – it is also worth ~0.27 GiB/thread. On-disk index bytes are IDENTICAL (byte-identity tested), wire format stays v8. ALTER EXTENSION pg_turbovec UPDATE TO '2.10.1';
2.9.0 2.10.0 none (no REINDEX) Phase Z5 Route A: an IVF index keeps its cell layout across INSERTs. Previously the first commit dropped the coarse/cell chains and the index became an O(n) flat scan until REINDEX. The flush now preserves them and the scan sweeps the appended tail exhaustively, so results stay EXACT. No wire change (stays v8) and no new meta field – the delta length is derivable as n_live - cell_directory.total_vectors(). Bounded by the new turbovec.ivf_max_delta_pct (default 10); past the bound it degrades and reports exactly as before, and = 0 restores pre-2.10.0 behaviour byte-for-byte. Also fixes an out-of-core bug where an appended tail was never gathered, making those rows unreachable (silent loss, not slowness). Measured 11% (in-memory) / 22% (OOC) median latency improvement at 1M x 256-d – NOT the 64x modelled, because a full 4-bit scan there is only 1.6x a 1-cell scan; this ships on the functional contract, not the latency delta. Corruption-validated under 6 writers + 4 readers + VACUUM for 5 min. Also documents that shared_preload_libraries = 'pg_turbovec' is required or every turbovec.* GUC silently vanishes. ALTER EXTENSION pg_turbovec UPDATE TO '2.10.0';
2.8.x 2.9.0 none (no REINDEX) Additive minor: one new function, plus a planner-cost fix. turbovec.index_degradation(regclass) quantifies an IVF degradation rather than merely flagging it (scan_fraction, est_slowdown = lists/probes, and the REINDEX command naming the index) – Phase Z1 made it observable and Phase Z4 made the planner cost it, but neither said how BAD it was, which is what decides whether to act. Phase Z4 also fixes three costing defects: IVF is now costed for the cells it PROBES (a degraded index is costed as flat, since that is the path it takes), index_selectivity comes from the planner’s own rel->rows / rel->tuples instead of a hardcoded 0.0, and a pre-existing unit error that had made a full 1M x 1024-d scan ~3000x too cheap (~23 vs PostgreSQL’s ~73,000 for the equivalent seq scan) is corrected. Wire format stays v8 – existing indexes decode byte-identically. ALTER EXTENSION pg_turbovec UPDATE TO '2.9.0';
2.8.3 2.8.4 none (no REINDEX) Two silent-diagnostic fixes; binary behaviour otherwise unchanged. (1) Phase Z1: an ordinary (TurboQuant) IVF index that takes writes degrades to a flat scan — that was always true, but the degradation was invisible because the deferred-commit flush planned its meta page with plan_with_blocked, which hardcodes lists: 0, erasing the IVF identity. It now preserves lists and stamps ivf_degraded, so turbovec.index_is_degraded() and the throttled ambeginscan WARNING tell the operator to REINDEX. Both are existing v4 meta fields — no wire change, no chain moved. (2) An INSERT into an index built WITH (assign_dups > 1) reported corrupt relfile pages: duplicate ids about a healthy index (soft assignment repeats ids across cells by design; turbovec_check verifies it clean and REINDEX cannot change it). It now reports FEATURE_NOT_SUPPORTED, names assign_dups, and says the index is effectively read-only — now documented in docs/BENCHMARKS.md and the reloption reference. Also adds the zvec source review + phases Z1–Z5 to docs/PARITY_GAPS.md. ALTER EXTENSION pg_turbovec UPDATE TO '2.8.4';
2.8.1 2.8.2 none (no REINDEX) Documentation: 4-bit IVF is supported — the “1-bit-only” benchmark result was about benefit, not support. Binary byte-identical to 2.8.1. A production user on bit_width = 4 read v2.8.0’s crossover result as a support restriction and asked for the feature; it has existed since v1.13.0. WITH (lists = N) composes with every bit_width — the only rejected combination is bit_width = 1 with graph = true. New § 0.6g answers a 4-bit user’s three questions separately (supported: yes; will it help: probably not, and this combination was never measured — 4-bit needs only a 32-wide rerank window vs 800 for 1-bit, so its scan is already cheap; what it costs: the per-probe recall ceiling and a 20.3 GB OOM-killed build). New docs/PRODUCTION.md section gives operators a decision table and a 20-minute experiment to settle it on their own data, comparing at matched recall. ALTER EXTENSION pg_turbovec UPDATE TO '2.8.2';
2.8.0 2.8.1 none (no REINDEX) Corrections to v2.8.0’s benchmark write-up; binary byte-identical. (1) The 1M corpus is not the same corpus as the 250k runs — the older Cohere dataset is now gated (HTTP 401), so the 1M arm used a different model/snapshot. Every cross-scale delta is relabelled suggestive, not measured; the within-run flat-vs-IVF conclusions are unaffected since both arms share one corpus. (2) A trap in the verification method: filtering to contention-unflagged rows is unsound if the baseline doesn’t survive the filter — on one arm all 8 flat baseline rows were flagged and zero survived, which would “prove” IVF wins against an empty set. The headline check was re-verified as valid, but partly by luck; rule added to docs/TESTING.md. (3) Restored the ground-truth-fix section lost to an earlier rewrite (the code fix was never affected), now including an accidental 1M validation: 3755.7 s → 275.4 s = 13.6× with 16 shared configs reproducing bit-identically. Build-memory figures relabelled as lower bounds. ALTER EXTENSION pg_turbovec UPDATE TO '2.8.1';
2.7.6 2.8.0 none (no REINDEX) The 1M IVF+BQ crossover measured on a real corpus; GT path parallelised. On 1M × 1024-d real Cohere data, WITH (lists = N, bit_width = 1) beats flat by 47 % at R@10 ≥ 0.90 and 38 % at ≥ 0.95, loses by 12 % at ≥ 0.98, and cannot reach ≥ 0.99 at all (per-probe ceiling 0.986). For bit_width ≥ 2 flat wins everywhere to 1M. Two-axis rule: 1-bit + n ≳ 1M + target ≲ 0.95 → lists = N; otherwise flat. Also: lists = 4096 is worse than lists = 1024 (11× build, ~50 % higher latency) so don’t exceed sqrt(n); IVF storage overhead halves at 1M (+3.1 %). The benchmark harness’s ground-truth query was parallelised in two steps (InitPlan constant, then CREATE TABLE AS instead of INSERT ... SELECT, since PostgreSQL won’t parallelise a data-writing statement) — ground truth verified row-for-row identical, so no published recall number moves. The v2.7.4 contended-latency caveat is resolved: re-run on a fixed host, recall reproduced exactly and the ratios held to two decimals. No wire change (stays v8). ALTER EXTENSION pg_turbovec UPDATE TO '2.8.0';
2.7.5 2.7.6 none (no REINDEX) Documentation consistency; discarded-1M root cause narrowed. Binary byte-identical to 2.7.5. The BQ docs still contained claims the measurements had falsified (BQ_RECALL_BENCH.md opened by saying it “contains no measurements”; ONEBIT_BQ.md had an orphaned “has NOT been run” fragment) — all swept and corrected. A competing diagnosis for the discarded 1M arm (an uncorrelated-subquery hoist making all 200 centres identical) was tested and ruled out: the hazard is real in the SQL and reproduces minimally, but the loaded corpus has 200 distinct centres and 9× cluster separation. The tie is inside each cluster — 5000 iid Gaussian points at d=768 concentrate, so a member’s 100 nearest span only 7.22 %. Cluster separation is not sufficient for rankability. Publishes the valid 1M storage/build numbers (bw2/bw1 = 2.000 exactly; 1-bit has the highest peak build RSS at 8.98 GiB because the corpus is read back resident to compute the mean), documents a harness GT plan pathology that cost 6.4 h, and makes the resolvability probe mandatory pre-flight for synthetic corpora. ALTER EXTENSION pg_turbovec UPDATE TO '2.7.6';
2.7.4 2.7.5 none (no REINDEX) 1-bit dimension sweep measured; a hi_dim_rerank doc error corrected. Binary byte-identical to 2.7.4. The sweep (256/512/1024-d) shows 1-bit’s penalty collapsing as dimension rises: the rerank window it needs versus 2-bit for R@10 ≥ 0.95 goes 125× → 25× → 8×, and its storage edge improves too (1.90× → 1.97×) as fixed per-index overhead amortises. 1-bit is a high-dimension technique — at 256-d it must rerank 6.4 % of the corpus for R@10 ≥ 0.99 and is effectively unusable; prefer 768-d and up. The 1024-d arm reproduced the published recall bit-identically at all 7 windows from a fresh database, validating both. Correction: the 1-bit hi_dim_rerank special case is a no-op at dim ≥ 256 (the auto window is identical for all bit widths there) and only widens below it — which strengthens the published comparisons, since they were quantizer-vs-quantizer rather than knob-vs-knob. Caveat: low dims are prefix slices, not native embeddings, so the trend is an upper bound. ALTER EXTENSION pg_turbovec UPDATE TO '2.7.5';
2.7.3 2.7.4 none (no REINDEX) Documentation accuracy + bench-harness isolation; binary byte-identical to 2.7.3. Corrects v2.7.3’s 1-bit latency figures, which were published without disclosing that the harness flagged all 24 rows as contended (the bench host cannot reach the 1.5 loadavg gate — an unrelated stuck process pins its idle floor near 2.0). Measured CPU busy on the pinned cores was only 16–26 %, so the ratios stand and the absolute ms are indicative; recall/storage/build are CPU-independent and unaffected. Also publishes the IVF+BQ measurements (+6.0 % storage over flat BQ; IVF imposes a per-probe-count recall ceiling a wider rerank window cannot break, so at 250k flat BQ dominates — a scale boundary, not a verdict), discards a synthetic 1M arm whose corpus was statistically unrankable (nn1→nn100 spread 6.6–10.4 % vs 37–268 % real), adds --run-id so concurrent bench arms cannot corrupt each other, corrects stale test counts (→ 427 passed / 8 ignored) and AGENTS.md’s migration matrix and wire version, and documents the high-dim IVF maintenance_work_mem OOM. ALTER EXTENSION pg_turbovec UPDATE TO '2.7.4';
2.7.2 2.7.3 none (no REINDEX) 1-bit empty-table insert fix + the measured BQ frontier. A bit_width = 1 index created on an empty table rejected its first INSERT (dim mismatch — index expects 0): the empty build stamps dim = 0 and the BQ insert path read the dim from the meta page instead of from the incoming row, the way the flat path always has. Found on a real corpus host, not by a test — every in-tree BQ fixture indexed an already-populated table. Also publishes the measured 1-bit frontier (arnold/AVX2, 250k × 1024-d Cohere-wiki, 100 held-out queries, exact ground truth): 3.98× smaller than 4-bit and 2.02× smaller than 2-bit, but 2.7–6.1× slower at matched recall and needing a 25× wider exact-rerank window to clear R@10 ≥ 0.99. All four pre-registered predictions held. (Correction added 2026-09-09: the latency rows are contention-flagged — the bench host cannot reach the harness’s load gate — so the ratios are the result and the absolute ms are indicative; recall/storage are unaffected. See docs/BQ_RECALL_BENCH.md § 0.) No wire change (stays v8). ALTER EXTENSION pg_turbovec UPDATE TO '2.7.3';
2.7.1 2.7.2 none (no REINDEX) Documentation-only: BUG#6 filed upstream. Binary byte-identical to 2.7.1. The root cause and one-line core fix verified in 2.7.1 are now reported on pgsql-hackers (2026-09-08). The filed patch’s code hunk is identical to the one verified here by A/B build, and its added core regression test was checked against both builds (ctid_matches 1 unpatched → 5 patched), so it genuinely gates the fix. docs/FILTERING.md and the tripwire test now carry the thread link; the test still asserts today’s broken behaviour and will fail when a fixed PostgreSQL reaches CI — that is the signal to relax the guidance. Nothing changes for users until then. ALTER EXTENSION pg_turbovec UPDATE TO '2.7.2';
2.7.0 2.7.1 none (no REINDEX) Documentation-only: BUG#6 root cause proven, core fix verified. Binary byte-identical to 2.7.0. SELECT ctid ... ORDER BY emb <=> q projecting (4294967295,0) was previously argued to be a PostgreSQL core bug; it is now demonstrated — reproduced on stock 18.4 with core GiST alone and zero turbovec loaded, traced line-by-line through indexam.c → nodeIndexscan.c → ExecForceStoreHeapTuple → slot_getsysattr, and the one-line core fix verified by building PostgreSQL both ways on one machine (ctid self-join 1→5, UPDATE ... WHERE ctid 1→5 rows, sentinel ctids 49/50→0/50). The affected function body is byte-identical across the 13.23–18.3 trees, so the fix applies to every supported major. Also verified that no query-level workaround exists (MATERIALIZED, text casts and subquery nesting all still yield the sentinel), which confirms the documented guidance — chain on your own key column, or use turbovec.knn() — is the only real answer. ALTER EXTENSION pg_turbovec UPDATE TO '2.7.1';
2.6.0 2.7.0 none for 2/¾-bit; REINDEX a 1-bit index that took inserts after a VACUUM IVF+BQ composition, a ~4.4× Hamming kernel, and two v2.6.0 BQ bugs fixed. WITH (lists = N, bit_width = 1) now works (cell-contiguous sign codes + coarse centroids + cell directory; BQ cells live in the RAW space, not TurboQuant’s rotated space). Two corruption-class bugs in v2.6.0’s BQ code are fixed: set_ivf_chains omitted bq_mean_count so an IVF+BQ build would have written the coarse chain on top of the corpus mean (the FOURTH occurrence of this bug class — v1.24.0 omitted graph_count, v2.6.0 fixed three sites); and flat-BQ aminsert neither re-persisted the tombstone bitmap (so every insert after a VACUUM resurrected every deleted row) nor de-duplicated by heap TID (so a re-insert added a second slot for the same row). Only bit_width = 1 indexes were affected. The Hamming kernel now counts 8 bytes at a time (measured 4.4–4.8× at embedding dims), with no unsafe and no CPU dispatch; an AVX2 variant was proven bit-identical and then declined on measurements. No wire change (stays v8). ALTER EXTENSION pg_turbovec UPDATE TO '2.7.0';
2.5.0 2.6.0 none (no REINDEX) 1-bit sign binary quantization works end to end. WITH (bit_width = 1) now builds, scans, inserts and vacuums (it previously ERRORed as “not yet implemented”). It is a distinct scheme from TurboQuant — turbovec rejects bit_width < 2 — storing dim/8 sign bits per vector with no scale, codebook, rotation or blocked chain, scored by Hamming and then exactly reranked by the AM. No wire-version bump: this is a new kind byte (KIND_BQ = 3), so every existing index keeps kind = SINGLE/COLBERT/GRAPH and decodes byte-identically, and the bq_mean_* fields occupy bytes that were reserved-and-zero on every prior version. Mean-centering is load-bearing (the naive sign-at-zero rule measured R@10 = 0.0 on dense-positive data), so the corpus mean is persisted and applied to queries; a corpus still collapsed after centering is rejected at build. Not composed with lists > 0 (IVF) in this release — that landed in 2.7.0. ALTER EXTENSION pg_turbovec UPDATE TO '2.6.0';
2.4.0 2.5.0 none (no REINDEX) Graph kind deprecated + Phase S-1 partition pruning. WITH (graph = true) now emits a deprecation WARNING (build-path removal scheduled, decode retained one further release with a REINDEX hint): measured at matched recall the graph loses on every axis — GIST-10M/960d R@10 ≥0.98 is reachable by IVF (28.4 ms, qps@8 161) and flat (34.2 ms, 31) but unreachable by the graph at any setting; it is also 57–90× slower to build with no out-of-core path. Use flat below ~1M and WITH (lists = N) at scale. turbovec.graph_ef, pack::repack and coarse_graph (which navigates centroids) are retained. Existing graph indexes keep working. Phase S-1 adds additive SQL for partition pruning at 1T scale — table turbovec.partition_summary plus refresh_partition_summary() and nearest_partitions(), which cut per-query fan-out from O(N) partitions to O(Kp). No index wire change (stays v8). ALTER EXTENSION pg_turbovec UPDATE TO '2.5.0';
2.3.0 2.4.0 none (no REINDEX) WAL amplification follow-up: stable chain starts. v2.3.0 stopped WAL-logging unchanged pages, but chain starts were packed back-to-back (scales_first = codes_first + codes_count), so any growth in the codes chain relocated every scales and ids page — and a relocated page must be rewritten. At 768d/4-bit a codes page holds just 21 rows, so nearly every flush paid to move those chains. The three growing chains are now padded to a multiple of PAD_PAGES (256), so a shift happens once per ~5400 rows instead of once per 21 (modelled: ~55 → ~5.7 KiB WAL/row at batch=512). Bounded slack of ≤6 MiB per index, independent of size; chains below one padding unit are not padded, so small indexes keep the previous layout byte-for-byte. No wire change (stays v8): *_count still means “blocks allocated”, and chain contents are located by *_first + n_vectors/rows_per_page, never by *_count. Existing indexes keep working; their chains relocate on the first full rewrite, safe by the v1.29.4 chains-before-meta invariant. ALTER EXTENSION pg_turbovec UPDATE TO '2.4.0';
2.2.2 2.3.0 none (no REINDEX) WAL amplification fix. A field report measured pg_turbovec inserts producing ~100% of all WAL on the host (1322 MB/25s with a single-row backfill running vs 31 kB/25s stopped, ~42,000x), with >99.98% invisible to pg_stat_statements because it is index maintenance rather than the INSERT. WAL per commit was ~constant at the index size (~500-750 MB on an 882 MB index) — every flush WAL-logged a full-page image of the whole relfile, ~4.3 TB/day, the dominant consumer of the host NVMe’s endurance. Since reconcile_flush_image appends new slots at the END and updates in place, the other pages were already byte-identical: a flush now compares and skips unchanged pages, so WAL scales with bytes changed instead of index size (the GenericXLog state is started lazily so an all-skipped batch emits no record). Also removed the dead write_chain() helper, which wrote pages with no WAL. Code-only: no wire change (stays v8), no SQL surface change, no REINDEX. Still batch your inserts — WAL scales with commits; see the new “WAL cost of inserts” section in docs/PRODUCTION.md. ALTER EXTENSION pg_turbovec UPDATE TO '2.3.0';
2.2.1 2.2.2+ none (no REINDEX to upgrade) turbovec_check() blind-spot fix. A field report found an IVF index (~2.18M vectors, 768d) where every KNN scan failed with turbovec’s InvalidScaleValue { slot: 1, value: -2.559434e22 } while turbovec_check() reported is_corrupt = false — so automated self-healing never fired and the index degraded silently until a manual REINDEX. The checker had deliberately skipped “the much larger codes/scales chains”, but the scales chain is the CHEAP one (one f32/vector: 8.7 MB vs the 17.4 MB ids chain it already read, vs 0.84 GB of codes) and is load-bearing for from_parts. It now validates every scale (finite, non-negative, sane magnitude) for every kind, in the same shared-lock snapshot as meta/ids, naming the failing slot in reason. The scan-path rejection is also now a real ERROR with a REINDEX INDEX hint instead of a bare Rust .expect() string. Code-only: no wire change (stays v8), no SQL surface change, no REINDEX to upgrade — but an already-damaged index still needs a one-time REINDEX INDEX <name>;, which this release will now actually report. ALTER EXTENSION pg_turbovec UPDATE TO '2.2.2';
2.2.0 2.2.1+ none (no REINDEX) Safety patch — the parallel graph build is now cancellable. v2.1.0’s interrupt hook lives in TLS and is invisible to rayon workers by design, but the driver also parks in rayon’s join for the whole parallel phase, so a pg_cancel_backend()/SIGINT against a 10M-node partitioned graph build was measured ignored for >13 minutes with 32 threads at 100% CPU while pg_stat_activity showed wait_event = NULL (it looked idle). Workers now poll an injected, thread-safe abort predicate and stop producing work; the driver raises PG’s real cancel error once the phase collapses. Code-only: no wire change (stays v8), no SQL surface change, no REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '2.2.1';
2.1.0 2.2.0+ none (no REINDEX) MINOR — graph-kind scan-beam retune. The graph kind’s scan-time beam width becomes its own knob, turbovec.graph_ef (int, Userset, default 0 = auto = 512, range 0..=1000000), and is DECOUPLED from turbovec.hi_dim_rerank. Through v2.1.0 the beam was (candidate_count * 4).max(64), so hi_dim_rerank = auto (whose real job is the flat/IVF exact-rerank window at dim >= 256) set the graph’s beam as a side effect — which showed up as an INVERTED recall cliff (128d, below the rerank threshold with the beam stuck at the 64 floor, was the WORST at R@10 0.720 while 512d looked perfect) and as a recall collapse at every dim with hi_dim_rerank = off. Recall was never dim-dependent, only beam-dependent. The 512 auto default comes from a measured 1M-scale recall-vs-latency frontier on SIFT-128 and GIST-960 (see docs/GRAPH_EF_BENCH.md): R@10 is monotone in the beam and plateaus by ~512 on both corpora. Pure scan-time change: no wire-format change (stays v8), no REINDEX — existing graph indexes pick up the new default immediately; flat/IVF indexes are untouched (no beam on those paths). ALTER EXTENSION pg_turbovec UPDATE TO '2.2.0'; + restart the backend (the GUC registers in _PG_init). ⚠️ One configuration regresses on recall, deliberately: a 960d graph index queried with hi_dim_rerank = auto (the default) goes R@10 0.971 → 0.920 / R@100 0.958 → 0.826, in exchange for p50 194 ms → 34 ms (5.6×). Those old numbers came from the undocumented side-effect beam of 3840 that auto bought at 960d — which vanished the moment anyone set hi_dim_rerank = off (0.840). SET turbovec.graph_ef = 3840 reproduces the pre-v2.2.0 auto beam exactly; = 128 reproduces the pre-v2.2.0 off beam at any dim. 128d indexes improve in both modes (R@10 0.976 → 0.990 at 1M) and lose nothing. Also newly documented: at 1M the flat kind beats the graph on both recall and latency on both corpora (SIFT-1M flat 0.993/0.96 ms vs graph 0.990/5.19 ms; GIST-1M flat 0.997/5.91 ms vs graph 0.920/34.6 ms) — the graph’s advantage is asymptotic and 1M is below the crossover on a 32-core AVX-512 host; its one measured win is concurrency scaling (1.8× vs 7.1× from 1→8 clients). See docs/GRAPH_EF_BENCH.md.
2.0.0 2.1.0+ none (no REINDEX) MINOR — graph-kind correctness. Fixes a CRITICAL graph concurrent-INSERT bug that could silently lose rows (the losing inserter’s rows vanished while turbovec_check() still said is_corrupt=false) as well as tear the adjacency chain; fixes graph_search returning fewer than k rows; makes a graph CREATE INDEX cancellable; and teaches turbovec_check() to validate the graph CSR adjacency. Also documents BUG#6 as an upstream PostgreSQL limitation (a kNN scan’s projected ctid is a sentinel — core’s ExecForceStoreHeapTuple never restores tts_tid; core GiST reproduces it without turbovec; patch in docs/upstream/). No wire change (stays v8), no REINDEX. ⚠️ turbovec_check() gains a trailing reason text OUT column, which changes its return type — CREATE OR REPLACE FUNCTION cannot do that, so the upgrade script DROPs and re-CREATEs the function. That is why this is a MINOR and not a patch. Anything selecting turbovec_check(...).* positionally should expect the extra column. ALTER EXTENSION pg_turbovec UPDATE TO '2.1.0'; + restart the backend.
1.x (any) → 2.0.0 2.0.0 ALTER EXTENSION then REINDEX INDEX <name>; once per index MAJOR: adopt upstream turbovec 1.0.0; wire format v7 → v8 (breaking). turbovec 1.0.0’s TQ+ per-coordinate calibration + v5 block-Hadamard rotation replaced the fork’s centroids/boundaries codebook + QR rotation, so every encoded byte differs — a pre-v8 index cannot be read in place. A pre-v8 (v1..v7) index is detected by MetaPageData::is_legacy_v7() and ambeginscan ERRORs at first scan with a REINDEX INDEX <name>; hint (never silent misread; validated end-to-end on EC2). Migration is REINDEX-from-heap: an in-place page converter was measured too lossy (−20.7 pp R@10 @ 4-bit, catastrophic @ 2-bit, from double quantization), so the index is rebuilt from the heap’s source vectors (full recall — the heap is the corpus, not a rebuild-from-external-corpus). Upside: materially faster (SIFT flat 9.6×, GIST IVF 5.6×, cold-scan 7.9×), same storage, recall matched-or-better, determinism intact, all corruption fixes re-proven on v8 (90-min A/B clean). ALTER EXTENSION pg_turbovec UPDATE TO '2.0.0'; + restart, then REINDEX INDEX <name>; per turbovec index.
1.29.6 1.29.7+ none v1.29.7 is a numerical-robustness patch: normalise_into (run on every indexed row) computed (1.0/norm) as f32, which overflowed to +inf for a tiny-nonzero-norm vector, poisoning every coordinate. Now divides per-element in f64. No wire change (v7), no SQL surface change, no REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.29.7';.
1.29.5 1.29.6+ none v1.29.6 is a Cargo.lock-only dependency-hygiene patch clearing all outstanding RustSec advisories: crossbeam-epoch 0.9.18→0.9.20 (RUSTSEC-2026-0204, via rayon) plus the tokio-postgres/postgres-protocol dev-dep advisories (via pgrx-tests only — not in the shipped .so). No source change, no wire change (stays v7), no SQL surface change, no REINDEX, byte-identical build output (gemm/turbovec/pgrx pins unchanged). ALTER EXTENSION pg_turbovec UPDATE TO '1.29.6';.
1.29.4 1.29.5+ none v1.29.5 is a production-hardening patch from a deep re-audit + at-scale stress test: fixes an IVF incremental-INSERT regression introduced in v1.29.4 (an ungated on-disk dup-id guard rejected soft-assigned IVF inserts — now gated to lists == 0), a multi-index partial-flush corruption-spreader (PreCommit now validates all dirty indexes before writing any), a corrupt/torn-meta unbounded-read + palloc guard in read_chain, SAVEPOINT-rollback persistence (new SubXactCallback), a VACUUM flat-shrink guard gap, and an empty-index KNN query error. No wire change (stays v7), no SQL surface change, no REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.29.5'; + restart. (Graph-kind issues remain tracked for a dedicated release — treat WITH (graph = true) as experimental.)
1.29.3 1.29.4+ none v1.29.4 fixes a VACUUM-INDEPENDENT .tvim corruption: the deferred aminsert PreCommit flush rewrote the relfile meta page BEFORE the row chains, so a pg_terminate_backend/cancel (e.g. a writer restart) landing mid-flush left meta.n_vectors advanced while the appended ids-chain slots were still zero — reloading as id 0 in more than one slot (XX001). Fixed by writing the CHAINS FIRST and the META LAST (build-path crash-safety invariant), plus a reconcile-flush dup-id guard that aborts (retryable) rather than entrench a pre-existing on-disk hole. No wire change (stays v7), no SQL surface change, no REINDEX to upgrade — existing indexes are read + written correctly in place. (An index ALREADY corrupted by an older binary should be REINDEXed once to clear the bad state.) A non-restarting long-lived single writer never triggers the bug — a valid immediate workaround. ALTER EXTENSION pg_turbovec UPDATE TO '1.29.4'; + restart.
1.29.2 1.29.3+ none v1.29.3 is a runtime-hardening patch from a full code audit: adversarial-input OOM guards on the sparsevec densify paths (::vector cast + sum(sparsevec) reject dim > 16000 before allocating), a clean ERROR (not a Rust panic across FFI) on a torn/corrupt scan slot lookup, and a check_for_interrupts!() in the VACUUM swap-remove loop so a large VACUUM stays cancellable. No wire change (stays v7), no SQL surface change, no REINDEX — all fixes are defensive guards in the binary. ALTER EXTENSION pg_turbovec UPDATE TO '1.29.3'; + restart. (A known CRITICAL follow-up NOT in this release: graph-kind incremental INSERT still has the pre-v1.29.2 lost-update window — keep graph inserts serial until the dedicated fix ships.)
1.29.1 1.29.2+ none v1.29.2 fixes the concurrent-VACUUM + deferred-flush data-corruption races (lost-update, VACUUM stale-snapshot, read_rotation/read_ids_only stale-meta torn reads) via reconcile-on-flush + full rewrite-lock discipline. Code-only, no wire change (stays v7), no SQL surface change, no REINDEX to upgrade – existing indexes are read + written correctly in place. (An index that ALREADY corrupted under an older binary should be REINDEXed once to clear the bad state.) ALTER EXTENSION pg_turbovec UPDATE TO 1.29.2; + restart the backend.
1.4.x – 1.26.x (ANY kind) 1.27.0+ REINDEX INDEX <name>; per index v1.27.0 (Phase Q-0) de-duplicates the on-disk quantized-codes storage, roughly halving the per-vector index footprint — the storage blocker cleared for large single-node indexes. Prior versions persisted each vector’s codes TWICE: the row-major bit-plane packed_codes chain AND the SIMD-blocked chain (pack::repack output). Since the blocked layout is a pure function of the packed codes, v7 drops the blocked chain from disk and recomputes it once per backend at index-open (per-query latency unchanged; scan results bit-identical). This bumps MetaPageData::version 6 → 7 and is NOT additive — a v7 relfile has no blocked chain, so it is not byte-compatible with any prior version for ANY kind (single-vector, ColBERT, IVF, graph); all kinds now emit v7 (the kind byte still discriminates). A pre-v7 index (v1..v6) is detected by MetaPageData::is_legacy_v6() and ambeginscan ERRORs with HINT: REINDEX INDEX <name>; at first scan (never silent corruption). No SQL-surface change. Migration: ALTER EXTENSION pg_turbovec UPDATE TO '1.27.0'; then REINDEX INDEX <name>; once per index. Until reindexed, scans ERROR with the hint (they do NOT return wrong results).
1.25.0 1.25.1+ none v1.25.1 is a release-tooling + docs/benchmark patch — no shippable code change (binary byte-identical to v1.25.0). Adds the tag-triggered PGXN + postgresql.org-news publish pipeline and the Qdrant/ANN-Benchmarks competitive benchmark (which validated v1.25.0’s hi_dim_rerank at scale: GIST-960-1M recall 0.876→0.953). No wire-format change (stays v6), no SQL-surface change, no REINDEX. ALTER EXTENSION pg_turbovec UPDATE TO '1.25.1'; is a no-op upgrade.
1.24.0 1.25.0+ none v1.25.0 adds turbovec.hi_dim_rerank (enum off|auto|on, default auto) — a dimension-aware exact-L2 rerank-window widening that recovers high-dimensional recall. The offline Gap-B investigation showed the high-dim gap (GIST-1M/960d ~0.86) is an in-cell quantized-RANKING loss, NOT a retrieval ceiling — the true NNs land in the probed cells (cell recall 0.98-0.996 at probes 64-128); a wider exact-L2 recheck recovers them (an SQ4 analog: R@10 0.666→0.978 at 960d). auto applies a clamp(dim, 256..=1024) candidate floor only for dim >= 256 (SIFT-128 untouched; an explicit search_k/oversample override past the floor wins). One new GUC, additive; no wire-format change (stays v6), no new operators/types/functions, no REINDEX. The new default improves high-dim recall out of the box at a small high-dim-only latency cost; SET turbovec.hi_dim_rerank = off restores exact pre-1.25.0 candidate behaviour. ALTER EXTENSION pg_turbovec UPDATE TO '1.25.0'; is sufficient.
1.23.0 1.24.0+ none v1.24.0 (Phase G-2b) adds VACUUM + incremental INSERT support for the graph index kind (WITH (graph = true)). Both previously raised a clear ERROR (v1.23.0 was build+scan only); they now work. NO wire-format change — wire format stays v6, byte-identical to v1.23.0; existing v4/v5/v6 indexes all decode unchanged, no REINDEX. VACUUM uses the same per-slot tombstone bitmap IVF already uses; aminsert is a deliberate O(n)-per-row whole-relfile rewrite (build-then-serve model; heavy churn should still REINDEX). Two real bugs fixed en route: a tombstone-chain/graph-adjacency-chain block-offset collision that corrupted a graph index on insert-after-VACUUM, and a VACUUM entry-point fallback that missed the “entry point survives but all its neighbors got tombstoned” dead-end. Both are binary fixes; neither changes the on-disk format. Still deferred: G-2c (SIMD traversal), G-2d (5M-scale HNSW-latency gate). ALTER EXTENSION pg_turbovec UPDATE TO '1.24.0'; is sufficient.
1.22.2 1.23.0+ none v1.23.0 adds WITH (graph = true), a new opt-in Vamana-style graph index kind (Phase G-2a). Wire format bumped to v6, ADDITIVE per kind — existing v4 (single-vector) and v5 (ColBERT) indexes decode byte-identical under the v6 binary (verified by dedicated tests). No REINDEX for any existing index. A graph index is a brand-new on-disk shape only a v6 binary produces; there is no in-place migration into it — build one explicitly. Correctness-first release: real Vamana build + scan with verified recall, but VACUUM/aminsert against a graph index raise a clear ERROR (not yet supported — rebuild after bulk changes), and the real HNSW-latency gate measurement has not yet been run (no latency/recall-vs-HNSW claim made). See CHANGELOG.md and . ALTER EXTENSION pg_turbovec UPDATE TO '1.23.0'; is sufficient.
1.4.x – 1.16.x (single-vector) 1.17.0+ none v1.17.0 (Phase F-2) adds the PERSISTENT ColBERT token index: a new vec_colbert_ops opclass over a turbovec.vector[] column builds a v5 on-disk token index, and turbovec.colbert_search reads stage-1 from the relfile instead of rebuilding a backend cache every call. The wire bump 4 → 5 is strictly additive per index kind: a single-vector index (vec_*_ops over a vector column) still emits wire version 4 with a zeroed kind byte (page offset 30) — its relfile is byte-identical to v1.16.0 (verified by the single_vector_still_emits_v4_bytes unit test and the v4_single_vector_index_byte_identical #[pg_test]). A v4 index decodes under the v5 binary as kind = KIND_SINGLE (the kind byte was a reserved zero on v4), so MetaPageData::is_legacy_v4() deliberately never trips and existing single-vector indexes need no REINDEX. Only an index built USING turbovec (col vec_colbert_ops) over a vector[] column is v5 (kind = KIND_COLBERT); that index is a brand-new shape (per-token slots, doc TID repeated in the ids chain) with NO ORDER BY semantics — ambeginscan ERRORs on an ORDER BY scan against it with a HINT to use turbovec.colbert_search. There is no in-place migration of a v4 single-vector index into a v5 ColBERT index; a ColBERT index is built fresh. ALTER EXTENSION pg_turbovec UPDATE TO '1.17.0'; registers the new opclass.
1.9.x (IVF minor, version TBD by the parent) none for flat indexes The IVF layer bumps MetaPageData::version 3→4 to add an opt-in inverted-file index, but the bump is flat-readable: a v3 index decodes under the v4 binary as a flat index (lists = 0), and MetaPageData::is_legacy_v3() deliberately never trips, so ambeginscan does NOT error on v3 or v4-flat indexes. Existing indexes keep working with no REINDEX — ALTER EXTENSION pg_turbovec UPDATE only. The new format is strictly opt-in: only an index built WITH (lists = N) (N > 0) uses the IVF layout (cell-contiguous codes + persisted coarse centroids + cell directory). A WITH (lists = 0) build (the default) is byte-identical to the v3 flat layout modulo the version byte. IVF-1 ships the build path + on-disk layout only; the scan path is still flat (cell-restricted search is IVF-2), so building WITH (lists > 0) today persists cells but does not yet change query latency. Recommended lists ≈ √n.

If you maintain pg_turbovec for a fleet of clusters, scripting the migration looks like:

DO $$
DECLARE
    idx record;
BEGIN
    FOR idx IN
        SELECT n.nspname || '.' || c.relname AS qname
        FROM pg_class c
        JOIN pg_am a ON a.oid = c.relam
        JOIN pg_namespace n ON n.oid = c.relnamespace
        WHERE a.amname = 'turbovec'
    LOOP
        RAISE NOTICE 'reindexing %', idx.qname;
        EXECUTE 'REINDEX INDEX CONCURRENTLY ' || idx.qname;
    END LOOP;
END $$;

REINDEX INDEX CONCURRENTLY rebuilds without taking an AccessExclusiveLock so reads keep working during the migration. The new index is built first; the cutover swap is atomic.

How the format-version contract is enforced

Three guardrails:

  1. src/index/page.rs::VERSION is the single source of truth. Any change to this constant in a patch release is a release-process bug and must be reverted before tagging.

  2. scripts/drift-check.sh includes a wire-format check. It reads the most recent tag’s VERSION constant from git history and compares it to the working tree. If the working tree has a higher VERSION than the last tag and the difference between tags is a patch bump, drift-check fails the push. (See § 11 of the script.)

  3. #[pg_test] wire_format_version_is_stable_across_patches in src/lib.rs reads the tag list from git, finds the most recent patch line (e.g. 1.3.0 → 1.3.x for any x), and asserts the compiled VERSION matches what the most-recent patch tag in that line emitted. Out-of-tree work (between tags) is allowed to bump freely; the gate is at tag time.

Adding a new minor release that bumps VERSION

Checklist for the release engineer (see RELEASING.md for the full release flow):

  1. Decide the new version layout. Add fields to MetaPageData if needed; bump VERSION to the next integer. Keep MIN_DECODE_VERSION at the oldest version still in production.
  2. Write the detection primitive. Add a method like MetaPageData::is_legacy_vN(&self) -> bool and a #[pg_test] relfile_legacy_vN_detection_primitive covering it.
  3. Wire the ERROR in ambeginscan. Match on the legacy-version detection and emit a pgrx::ereport!(ERROR, FEATURE_NOT_SUPPORTED, "this index was built under pg_turbovec ≤ X.Y; run \REINDEX INDEX
  4. Add a row to the migration matrix above.
  5. Add a migrations/0NN_pg_turbovec_vX.Y.0.sql if there’s any SQL-level change (drop a table, rename a function, etc).
  6. CHANGELOG entry must include a “Migration” section with the exact REINDEX scripts users need to run.
  7. scripts/drift-check.sh will start nagging until VERSION updates land alongside the version bump in Cargo.toml. That’s intentional — both must change together.

What “patch release” actually means

A patch release fixes a bug, plugs a security hole, or improves performance without changing the on-disk format or the SQL surface. Patch releases include:

  • Bug fixes that don’t change the page layout.
  • Performance improvements to the search kernel that produce bit-identical scoring (e.g. SIMD width upgrade, allocator tuning).
  • New SQL helper functions that don’t change existing ones.
  • Documentation, CI, and bench-script changes.

Patch releases explicitly DO NOT include:

  • Changes to MetaPageData field order or sizes.
  • New MetaPageData fields (those bump VERSION and need a minor).
  • Changes to the page-allocation layout (chain offsets, rows_per_*_page).
  • Changes to the SIMD-blocked layout produced by pack::repack (would change blocked_codes_first / _count semantics).
  • Changes to the codebook serialisation.
  • Changes to existing operator definitions (<=>, <#>, etc.) or function signatures.

If a “bug fix” requires touching any of those, it’s a minor release, not a patch.