pgvector Index Benchmark
Which Postgres vector index should you actually ship? Measured, not guessed.
pgvector's IVFFlat and HNSW and pgvectorscale's StreamingDiskANN all promise fast similarity search, but vendor charts rarely compare them at matched recall, on real data, under memory pressure. I built a reproducible harness that answers the question with numbers: build time, disk size, latency percentiles, recall@10 and behaviour under 100 concurrent clients.
Impact
- HNSW p50 1.59 ms vs 144.9 ms seq-scan
- Faster than exact search
- 91×
- HNSW at ef_search = 40
- Recall@10 kept
- 0.972
- DiskANN 179 MB vs HNSW 673 MB
- Smaller index
- 3.7×
- 50K Wikipedia articles · 384-dim
- Vectors benchmarked
- 328K
01/Architecture
How the pieces fit.
Six file-backed stages feed a Postgres 17 container; the benchmark loop builds, measures and drops one index at a time.


02/By the numbers
What the data says.
Serial throughput
Queries per second, one client, 328,587 vectors. Higher is better.
- exact6.9 qps
- ivfflat184 qps
- diskann242 qps
- hnsw570 qps
Index size on disk
Steady-state footprint at the same tier. Lower is better.
- hnsw673 MB
- ivfflat540 MB
- diskann179 MB
Memory squeeze: HNSW ÷ DiskANN build time
80,923 rows, shrinking maintenance_work_mem. 1.0 would be break-even — the gap closes, then turns.
- 2 GB0.15
- 96 MB0.47
- 8 MB0.75
- 2 MB0.72
03/How it works
4 stages, each with a job.
stage 01
Data prep
- 01Stream 200 / 5,000 / 50,000 Wikipedia articles into three scale tiers.
- 02Chunk to 180 words with 40 overlap — every chunk fits MiniLM's 256-token window, so nothing is silently truncated.
- 03Embed with all-MiniLM-L6-v2 (384-dim, L2-normalised) into a memory-mapped .npy, so tiers larger than RAM still work.
04/Evidence
Straight from the repo.


Recall vs latency
328,587 rows: HNSW reaches 0.972 recall@10 at 1.59 ms p50.


Build & storage cost
DiskANN builds slowest but stores the smallest index.
05/Engineering decisions
What I optimised for.
01
Recall is an operating point
Each index is measured at one tuning setting and compared only at matched recall — a single bar chart flatters whichever index is tuned loosest.
02
Honest queries
Query vectors are held out of the load, so the benchmark never retrieves a vector it was handed verbatim.
03
Isolation per index
Build, measure, drop. Each index gets the cache to itself, and inserts are timed on an unindexed table.
04
Every stage re-runnable
Stages talk only through files under data/<tier>/, so any one of them can be re-run alone — or driven end to end from a notebook.
06/Stack
Built with.
Next case study
Data Feed Engine
A new data feed should be a config file, not a new pipeline.