pg_sift 3.5.1: fixes a crash of a backend (signal 11) when a parallel aggregate scan with a correlated lookup is planned with workers and none can be launched (the pool of parallel workers is spent). Update with the files of this release and restart the server; no ALTER EXTENSION step, no REINDEX.

Highlights

  • Fixed: a crash of a backend when no parallel worker can be launched. A Custom Scan (SiftAgg) planned with Workers Planned above 0 and a correlated child (a LEFT JOIN LATERAL (..) LIMIT 1) keeps pointers into the parallel shared memory. When LaunchParallelWorkers starts no worker (max_parallel_workers or max_worker_processes is spent by other sessions), PostgreSQL destroys the parallel context, the leader scans alone and read the unmapped segment: the backend ended with signal 11 and the server restarted the others (on a hot standby too). The scan drops the pointers before the context is destroyed. Test agg_par_no_workers (signal 11 on 3.5.0). The same code is in 3.4.0 to 3.5.0 (not run on 3.4.x); earlier releases that contain it are not patched: upgrade to 3.5.1.

Who is affected and what to do

  • Every install of 3.5.0, and of 3.4.0 to 3.4.2: the statement above, only when the workers of the statement cannot be started. Install 3.5.1 and restart the server. A backend that already loaded the old library keeps it until its connection ends, and the crash is in that library: after the install restart the server (a standby too), or reconnect every pool and worker.
  • Until then: ALTER ROLE app SET max_parallel_workers_per_gather = 0, or raise max_worker_processes and max_parallel_workers so that the pool is not spent.
  • Nothing else changes: plans, answers and formats are those of 3.5.0.

Compatibility

  • PostgreSQL 17, 18 and 19; the packages below are for PostgreSQL 18 (other majors build from source with make USE_PGXS=1). A drop-in replacement of 3.5.0: extension SQL 3.5 (ALTER EXTENSION pg_sift UPDATE has nothing to do and there is no pg_sift--3.5--3.5.1.sql), no change of the BM25, sift_col or sift_table formats, no setting changes its default. pg_get_loaded_modules() on PostgreSQL 18 says 3.5.1+<commit>; the extension version stays 3.5.

Packages

x86-64 packages for PostgreSQL 18 are attached below (SHA256SUMS covers every asset); other versions build from source with make USE_PGXS=1. The packages depend on liblz4 and libzstd.

  • Debian 13, Ubuntu 24.04 (PGDG postgresql-18): sudo apt install ./postgresql-18-pg-sift_3.5.1-1_amd64.deb
  • RHEL, Rocky, Alma 9 and 10 (PGDG postgresql18-server): sudo dnf install ./pg_sift_18-3.5.1-1.el9.x86_64.rpm
  • Arch Linux: AUR pg_sift.

Every change: CHANGELOG.md.

SQLite extension

pg_sift for SQLite 3.5.1 (sift, a loadable extension; a separate component from the PostgreSQL extension above). Rebuilt with the PostgreSQL 3.5.1 release: no change to the SQL interface or the shared format.

  • No change: the fixes of this release are in the PostgreSQL column index, the aggregate planner and sift_table, which are not in this extension. No speed-up or slow-down is claimed.
  • Needs SQLite 3.38 or newer; Linux x86-64, glibc 2.34 or newer. Built on Debian 12, loaded into the SQLite of Debian 12, Ubuntu 24.04 and AlmaLinux 10 before release.
  • Storage format 2: tables of 3.4.0 and 3.5.0 open as they are, nothing to rebuild.

Download sift-sqlite-3.5.1-linux-x86_64.tar.gz (below, covered by SHA256SUMS with the packages), unpack it and .load ./sift from the sqlite3 shell (sha256sum --ignore-missing -c SHA256SUMS). Documentation: sqlite/README.md. Every change: sqlite/CHANGELOG.md.

MariaDB extension

pg_sift for MariaDB 3.5.1 (sift, a UDF library and two SQL files; a separate component from the extensions above). Rebuilt with the PostgreSQL 3.5.1 release: no change to the SQL files.

  • No change: the fixes of this release are in the PostgreSQL column index, the aggregate planner and sift_table, which are not in this extension. No speed-up or slow-down is claimed.
  • One library per MariaDB series, because it calls the server’s own symbols and refuses to load into another series: 10.11, 11.4, 11.8, 12.1 and 13.0; Linux x86-64. Each was built in the stock mariadb:<series> image against that server’s headers and loaded into a server of the same image before release. BUILD-INFO in the tarball lists the server version and the glibc each library needs; a server of another distribution or of a series not listed is built from source (make -C mariadb).
  • Index format 2: an index of 3.4.0 and 3.5.0 opens as it is, nothing to rebuild.

Download sift-mariadb-3.5.1-linux-x86_64.tar.gz (below, covered by SHA256SUMS with the packages), unpack it, copy lib/<series>/libsift_udf.so of your server’s series (SELECT VERSION()) into its plugin_dir, then SOURCE sql/functions.sql once per server and sql/sift.sql once per database (sha256sum --ignore-missing -c SHA256SUMS). Documentation: mariadb/README.md. Every change: mariadb/CHANGELOG.md.