pg_sift 1.2.0: phrases decided from the index, faster ranked queries, and PostgreSQL 17 support. Install the new files and REINDEX your sift indexes to get word positions; no ALTER EXTENSION step is needed (the SQL interface is unchanged).

Highlights

  • Phrase queries from the index. Word positions are now stored by default (positions = on), so phrases and count(*) of phrases no longer re-read heap text. On 218k Wikipedia articles: phrase 3.3x and count(*) 4.7x faster than pg_search, and the default index is 131 MiB, 1.4x smaller than pg_search’s 184 MiB. positions = off keeps the small 52 MiB index (3.5x smaller than pg_search) and the heap recheck.
  • Faster ranked search. New top-k kernels (AVX-512, AVX2, scalar) score only documents that can still reach the top k: ranked AND/OR use about 27% fewer cycles, count(*) of OR queries is about 27% faster, results and ties are bit-identical.
  • Better filter planning. A ranked scan with a selective filter now picks the pushed-down plan when it is cheaper (418 ms to 5 ms in the reported case), including long = ANY(ids) lists.
  • Prefix queries never fail. A prefix with more than sift.max_expansions terms (h* on a large table) matches everything it should instead of raising an error; prefix_index is now a speed/size choice, not a requirement.
  • Fixes. sift.score() through joins, field qualifiers (name:(a b)) in groups, wrong answers when several sift indexes share a key, planner crash and assertion failures.

Compatibility

  • PostgreSQL 17 and 18 are supported; PostgreSQL 19 is experimental (CI builds it from git on every push and it passes).
  • Indexes created by this version use on-disk format 1.1 and cannot be opened by earlier releases. Format 1.0 indexes keep working, without positions, until REINDEX.
  • With positions = off, the index is about 10% larger than in 1.1.0 (52.5 against 47.9 MiB) in exchange for ranked AND 1.6x, OR 1.9x and filtered queries 1.8x faster.

Packages

x86-64 packages for PostgreSQL 18 are attached below (SHA256SUMS included); other versions build from source with make USE_PGXS=1.

  • Debian 13, Ubuntu 24.04 (PGDG postgresql-18): sudo apt install ./postgresql-18-pg-sift_1.2.0-1_amd64.deb
  • RHEL, Rocky, Alma 9 and 10 (PGDG postgresql18-server): sudo dnf install ./pg_sift_18-1.2.0-1.el9.x86_64.rpm
  • Arch Linux: AUR pg_sift.

Every change: CHANGELOG.md.