RK
back to work
Case studyVector search · PostgreSQL2026

pgvector Index Benchmark

Which Postgres vector index should you actually ship? Measured, not guessed.

pgvector's IVFFlat and HNSW and pgvectorscale's StreamingDiskANN all promise fast similarity search, but vendor charts rarely compare them at matched recall, on real data, under memory pressure. I built a reproducible harness that answers the question with numbers: build time, disk size, latency percentiles, recall@10 and behaviour under 100 concurrent clients.

Impact

HNSW p50 1.59 ms vs 144.9 ms seq-scan
Faster than exact search
91×
HNSW at ef_search = 40
Recall@10 kept
0.972
DiskANN 179 MB vs HNSW 673 MB
Smaller index
3.7×
50K Wikipedia articles · 384-dim
Vectors benchmarked
328K

01/Architecture

How the pieces fit.

pgvector-architecture.html open full screen
pgvector Index Benchmark architecture diagram

02/By the numbers

What the data says.

Serial throughput

Queries per second, one client, 328,587 vectors. Higher is better.

  • exact6.9 qps
  • ivfflat184 qps
  • diskann242 qps
  • hnsw570 qps

Index size on disk

Steady-state footprint at the same tier. Lower is better.

  • hnsw673 MB
  • ivfflat540 MB
  • diskann179 MB

Memory squeeze: HNSW ÷ DiskANN build time

80,923 rows, shrinking maintenance_work_mem. 1.0 would be break-even — the gap closes, then turns.

  • 2 GB0.15
  • 96 MB0.47
  • 8 MB0.75
  • 2 MB0.72

03/How it works

4 stages, each with a job.

stage 01

Data prep

  1. 01Stream 200 / 5,000 / 50,000 Wikipedia articles into three scale tiers.
  2. 02Chunk to 180 words with 40 overlap — every chunk fits MiniLM's 256-token window, so nothing is silently truncated.
  3. 03Embed with all-MiniLM-L6-v2 (384-dim, L2-normalised) into a memory-mapped .npy, so tiers larger than RAM still work.

04/Evidence

Straight from the repo.

Recall vs latency

Recall vs latency

328,587 rows: HNSW reaches 0.972 recall@10 at 1.59 ms p50.

Build & storage cost

Build & storage cost

DiskANN builds slowest but stores the smallest index.

05/Engineering decisions

What I optimised for.

01

Recall is an operating point

Each index is measured at one tuning setting and compared only at matched recall — a single bar chart flatters whichever index is tuned loosest.

02

Honest queries

Query vectors are held out of the load, so the benchmark never retrieves a vector it was handed verbatim.

03

Isolation per index

Build, measure, drop. Each index gets the cache to itself, and inserts are timed on an unindexed table.

04

Every stage re-runnable

Stages talk only through files under data/<tier>/, so any one of them can be re-run alone — or driven end to end from a notebook.

06/Stack

Built with.

Python 3.12uvPostgreSQL 17pgvectorpgvectorscalesentence-transformersHugging Face datasetspsycopg 3NumPymatplotlibDocker Compose

Next case study

Data Feed Engine

A new data feed should be a config file, not a new pipeline.