bloompg 0.1.2

This Release
bloompg 0.1.2
Date
Status
Stable
Abstract
Adaptive Bloom-filter transfer for PostgreSQL
Description
BloomPG executes selected relation inputs during planning, propagates exact Bloom or bitmap filters through an equality-join graph, and gives reduced relations back to PostgreSQL for a second planning pass.
Released By
yiqiao
License
MIT
Resources
Special Files
Tags

Extensions

bloompg 0.1.2
Adaptive Bloom-filter transfer for PostgreSQL

Documentation

CONFIGURATION
Configuration
CONTRIBUTING
Contributing to BloomPG
SECURITY
Security policy
CHANGELOG
Changelog

README

BloomPG

PostgreSQL 18 Version 0.1.2 CI License: MIT

BloomPG accelerates complex analytical joins in PostgreSQL 18. It propagates Bloom or exact bitmap filters across equality joins, materializes reduced inputs, and lets PostgreSQL plan and execute the smaller join problem.

It is most useful when a selective table can eliminate a large fraction of one or more joined tables. Existing SQL does not need to change, and unsupported queries continue to use PostgreSQL’s native planner.

Results

PostgreSQL 18.4 on two Intel Xeon Platinum 8474C CPUs, using the SQL corpora from DataFusion-Bloom. Both sides use the same 16-worker global and per-Gather ceiling. BloomPG also uses 16 transfer workers, the default ndv progress metric, and a 2 GB materialization budget. Times include planning, transfer, materialization, execution, and complete output consumption.

Workload PostgreSQL storage Completed Native PG BloomPG Total speedup
CEB IMDB 9.43 GB, indexed 3,132/3,133 15,826.369 s 3,822.980 s 4.140x
JOB 9.43 GB, indexed 113/113 217.753 s 69.337 s 3.141x
STATS-CEB 113 MB, indexed 145/146 697.363 s 237.348 s 2.938x
TPC-H SF10 22.02 GB, indexed 22/22 119.082 s 114.413 s 1.041x

All completed Native/BloomPG pairs produced identical complete-output fingerprints. Each execution has a 300-second limit, and totals include only queries completed by both systems. BloomPG completed every query; native PostgreSQL reached the limit once in CEB and once in STATS-CEB. PostgreSQL’s normal indexes remain available to both sides, while BloomPG’s experimental index-assisted transfer materialization is disabled for this table.

Install

BloomPG currently requires PostgreSQL 18 server headers and PGXS.

Docker quickstart

Build PostgreSQL 18 with BloomPG, create a small star schema, and compare the same five-table analytical query with native PostgreSQL and BloomPG:

docker compose up --build --abort-on-container-exit --exit-code-from demo demo

The demo uses query-local instant sampling and prints both complete query timings and BloomPG’s transfer trace for an ordinary five-table analytical query under PostgreSQL’s default optimizer settings. Remove the demo database afterward with docker compose down --volumes. To open an interactive SQL prompt while the database is running, use:

docker compose exec db psql -U postgres -d bloompg_demo

Build from source

make PG_CONFIG=/path/to/postgresql-18/bin/pg_config
make PG_CONFIG=/path/to/postgresql-18/bin/pg_config install

Load the module in every backend, normally through postgresql.conf:

shared_preload_libraries = 'bloompg'

Restart PostgreSQL, then create the extension in each database:

CREATE EXTENSION bloompg;
SELECT bloompg_version();

To upgrade from 0.1.0 or 0.1.1 after installing the new files:

ALTER EXTENSION bloompg UPDATE TO '0.1.2';

Use

BloomPG is enabled by default after it is loaded. Run analytical queries as usual:

SELECT count(*)
FROM fact
JOIN dimension USING (dimension_id)
WHERE dimension.region = 'APAC';

The main controls are:

SET bloompg.enable = on;
SET bloompg.transfer_progress_metric = 'ndv'; -- default; use 'rows' for compatibility
SET bloompg.transfer_workers = 8;

ndv tracks only exact equality-key cardinality and is the recommended default. rows keeps the original row-count propagation algorithm. See configuration for all settings and memory guidance. Exact-key index-assisted materialization is available as an experimental, opt-in feature through bloompg.index_transfer; stable transfer uses scan paths by default.

For query-level diagnostics:

SET bloompg.profile = on;
SET bloompg.profile_log = off;

-- Run the query, then inspect the latest profile.
SELECT jsonb_pretty(bloompg_last_profile());
SELECT bloompg_last_trace();

Scope

  • PostgreSQL 18 only; developed and tested with PostgreSQL 18.4 on Linux.
  • Read-only queries with strict, hashjoinable equality joins.
  • Inner, outer, semi, anti, composite, self, prepared-parameter, CTE, subquery, view, and supported partitioned-relation shapes.
  • Modifying statements, row locking, row-level security, volatile expressions, and unsupported scopes bypass BloomPG.

BloomPG performs real scans during planning. Its query-wide materialization budget defaults to one eighth of detected host/cgroup memory, capped at 2GB, and falls back to the native plan when full. Size it for expected analytical concurrency. BloomPG is intended for controlled analytical workloads, not as an unreviewed default for a multi-tenant OLTP cluster.

Verify a build

make PG_CONFIG=/path/to/postgresql-18/bin/pg_config installcheck \
  PGHOST=/tmp PGPORT=5432
make pythoncheck PG_CONFIG=/path/to/postgresql-18/bin/pg_config

See CONTRIBUTING.md for development and performance-test guidelines. Report security issues privately as described in SECURITY.md.

Related projects

Bloom, BloomPG, and Bloom for Apache DataFusion are sibling projects exploring robust predicate transfer across different query engines.

References

License

BloomPG is available under the MIT License.