Contents
Benchmarks
Harnesses, configs, dataset scripts, the fake Pinecone and the write-ups behind pg_pinecone’s performance numbers. Nothing here is
built or run by CI: .github/workflows/ci.yml formats and lints the Rust crates and runs the Rust tests only.
| dir | measures |
|---|---|
vector/ |
Filtered and unfiltered ANN search: pgvector hnsw / ivfflat / halfvec against pg_pinecone’s native local engine and the remote engine; residual WHERE filters and the iterative scan, on synthetic data, SIFT1M, wiki_dpr 768d and Cohere wikipedia-22-12 768d |
Common setup
- State, downloads and scratch clusters live in
bench/.data/(gitignored). Move them withPG_PINECONE_BENCH_DATA=/path. - The Postgres under test is
PG_PINECONE_BENCH_PG(an install prefix withbin/pg_ctl).vector/defaults to a pgrx-managed PG 17 (~/.pgrx/17.10/pgrx-install) that has pgvector. pgrx’s own PG is a--enable-cassertbuild: fine for plumbing and recall, not for timing. - Build pg_pinecone into a private pgrx home (
PGRX_HOME), never into one other worktrees install into. - The smoke below needs no Pinecone key and runs with
env -u PINECONE_API_KEY. The remote configs read a key from a file and spend real money (seevector/box/README.md). - The Python venv is
vector/.venv(verified with Python 3.14). Paths must not contain spaces.
Smoke command (verified 2026-10-12; sizes and timings in vector/README.md)
From bench/vector (PG_PINECONE_BENCH_PG needs pgvector and pg_pinecone installed: see “Install pg_pinecone” in vector/README.md):
python3 -m venv .venv && .venv/bin/pip install -r requirements.txt
export PG_PINECONE_BENCH_DATA="$(cd .. && pwd)/.data" PG_PINECONE_BENCH_PG=/path/to/postgres-install-prefix
scripts/download-datasets.sh sift # 168 MB; `wikidpr` is 5.3 GB
env -u PINECONE_API_KEY .venv/bin/python run.py --config configs/smoke-native.yaml \
--data-dir "$PG_PINECONE_BENCH_DATA/vector/pgdata-smoke" -v # 50k x 768d, 411 s
Results land in vector/results/ (one JSON and one markdown file per run), gitignored. The committed write-ups are the records of past runs:
vector/REPORT-residual-filter.md: iterative scan vs pgvector, 1M x 768d, 2026-10-07.vector/REPORT-halfvec.md: pgvector halfvec vs FP32 hnsw and native, 1M x 768d, 2026-10-16.vector/REPORT-halfvec-filtered.md: the same on 1% and 10% filters, 0.10.0, 2026-10-22.vector/REPORT-overram.md: search under a memory cap, 2026-10-22.
Not reproducible from this tree
- Anything against real Pinecone needs a paid Pinecone project (and, for the box recipe, AWS): the
remote_*configs and their phases invector/box/phase.sh. - Absolute numbers. Reports quote a laptop (M4 Max, 36 GB, other apps running) or an AWS box; compare rows inside a table.