Inference
Antfly runs extraction, embedding, decision, generative, speech and OCR models on its own GPU kernels. These plots compare speed and peak memory on CUDA and Metal with each model’s reference runtime, for models that Antfly Cloud also hosts.
Antfly inference · CUDA
NVIDIA L4 GPU. Same weights on both sides; each mark is one workload. Measured on Antfly v0.3.1.
GLiNER2.5 base (vs Fastino gliner2 2.1.0): Speed 7/7, memory 7/7 complete. speed sentence, Antfly 2.90× faster; paragraph, Antfly 1.92× faster; 350-word page, Upstream 1.13× faster; Wikipedia paragraph, Antfly 1.72× faster; entities + classes + relations, Antfly 1.84× faster; 8 paragraphs per request, Upstream 1.12× faster; 32 paragraphs per request, Antfly 1.04× faster. memory sentence, Antfly 1.72× smaller memory footprint (Antfly 1.77 GB vs upstream 3.06 GB); paragraph, Antfly 1.75× smaller memory footprint (Antfly 1.77 GB vs upstream 3.09 GB); 350-word page, Antfly 1.75× smaller memory footprint (Antfly 1.77 GB vs upstream 3.10 GB); Wikipedia paragraph, Antfly 1.53× smaller memory footprint (Antfly 2.04 GB vs upstream 3.11 GB); entities + classes + relations, Antfly 1.81× smaller memory footprint (Antfly 1.77 GB vs upstream 3.20 GB); 8 paragraphs per request, Antfly 1.96× smaller memory footprint (Antfly 1.86 GB vs upstream 3.64 GB); 32 paragraphs per request, Antfly 2.65× smaller memory footprint (Antfly 1.86 GB vs upstream 4.93 GB). EmbeddingGemma 2 (vs sentence-transformers 6.2.0): Speed 7/7, memory 7/7 complete. speed sentence, Antfly 5.48× faster; 32 sentences per request, Upstream 2.51× faster; paragraph, Antfly 3.70× faster; 32 paragraphs per request, Upstream 1.39× faster; 8 pages per request, Upstream 1.56× faster; ~2K-token document, Upstream 2.50× faster; ~7K-token document, Upstream 4.59× faster. memory sentence, Antfly 1.17× smaller memory footprint (Antfly 3.70 GB vs upstream 4.34 GB); 32 sentences per request, Antfly 1.33× smaller memory footprint (Antfly 3.69 GB vs upstream 4.91 GB); paragraph, Antfly 1.20× smaller memory footprint (Antfly 3.69 GB vs upstream 4.42 GB); 32 paragraphs per request, Antfly 1.44× smaller memory footprint (Antfly 3.70 GB vs upstream 5.34 GB); 8 pages per request, Antfly 1.44× smaller memory footprint (Antfly 3.68 GB vs upstream 5.32 GB); ~2K-token document, Antfly 1.21× smaller memory footprint (Antfly 3.69 GB vs upstream 4.44 GB); ~7K-token document, Antfly 1.67× smaller memory footprint (Antfly 3.68 GB vs upstream 6.13 GB). GLiNER2.5-Decide (vs Fastino gliner2 2.1.0): Speed 3/3, memory 3/3 complete. speed 1 review per request, Antfly 2.44× faster; 8 per request, Antfly 1.15× faster; 32 per request, Antfly 1.10× faster. memory 1 review per request, Antfly 1.21× smaller memory footprint (Antfly 3.96 GB vs upstream 4.80 GB); 8 per request, Antfly 1.26× smaller memory footprint (Antfly 3.96 GB vs upstream 5.00 GB); 32 per request, Antfly 1.45× smaller memory footprint (Antfly 3.96 GB vs upstream 5.73 GB). Gemma 4 E4B (vs llama.cpp b10809): Speed 5/5, memory 5/5 complete. speed short prompt, 256 tokens out, Antfly 1.04× faster; 350-word page, 128 out, Antfly 1.06× faster; ~2K-token prompt, 64 out, Antfly 1.12× faster; image, 64 out, Antfly 1.07× faster; 5–15 s audio, 64 out, Antfly 1.05× faster. memory short prompt, 256 tokens out, Antfly 1.22× smaller memory footprint (Antfly 7.07 GB vs upstream 8.60 GB); 350-word page, 128 out, Antfly 1.06× smaller memory footprint (Antfly 8.11 GB vs upstream 8.60 GB); ~2K-token prompt, 64 out, Upstream 1.00× smaller memory footprint (Antfly 8.61 GB vs upstream 8.60 GB); image, 64 out, Antfly 1.06× smaller memory footprint (Antfly 8.11 GB vs upstream 8.62 GB); 5–15 s audio, 64 out, Antfly 1.07× smaller memory footprint (Antfly 8.06 GB vs upstream 8.60 GB). Same prompt token count on all 46 inputs; generated text matches llama.cpp on none. Not identical outputs.. Whisper small (vs Transformers 5.23.0): Speed 3/3, memory 3/3 complete. speed 5.9 s clip, Antfly 2.91× faster; 15.1 s clip, Antfly 3.54× faster; 29.4 s clip, Antfly 4.11× faster. memory 5.9 s clip, Antfly 1.56× smaller memory footprint (Antfly 2.13 GB vs upstream 3.33 GB); 15.1 s clip, Antfly 1.55× smaller memory footprint (Antfly 2.19 GB vs upstream 3.39 GB); 29.4 s clip, Antfly 1.58× smaller memory footprint (Antfly 2.14 GB vs upstream 3.39 GB). LightOnOCR-3 0.8B (vs Transformers 5.23.0): Speed 1/1, memory 1/1 complete. speed one page, Antfly 2.60× faster. memory one page, Antfly 1.24× smaller memory footprint (Antfly 3.82 GB vs upstream 4.73 GB)
- Speed: upstream time ÷ Antfly time
- Peak memory: upstream ÷ Antfly
- Hollow: upstream better on that workload
Speed by model · CUDA
- Model
- GLiNER2.5 base
- Compared against
- Fastino gliner2 2.1.0
- Typical workload
- 1.72× faster
- Range
- 1.13× slower to 2.90× faster
- Model
- EmbeddingGemma 2
- Compared against
- sentence-transformers 6.2.0
- Typical workload
- 1.56× slower
- Range
- 4.59× slower to 5.48× faster
- Model
- GLiNER2.5-Decide
- Compared against
- Fastino gliner2 2.1.0
- Typical workload
- 1.15× faster
- Range
- 1.10× to 2.44× faster
- Model
- Gemma 4 E4B
- Compared against
- llama.cpp b10809
- Typical workload
- 1.06× faster
- Range
- 1.04× to 1.12× faster
- Model
- Whisper small
- Compared against
- Transformers 5.23.0
- Typical workload
- 3.54× faster
- Range
- 2.91× to 4.11× faster
- Model
- LightOnOCR-3 0.8B
- Compared against
- Transformers 5.23.0
- Typical workload
- 2.60× faster
- Range
- One workload
Antfly inference · Metal
Apple M4 Max GPU. Same weights on both sides; each mark is one workload. Measured on Antfly v0.3.1.
GLiNER2.5 base (vs Fastino gliner2 2.1.0): Speed 7/7, memory 7/7 complete. speed sentence, Antfly 1.83× faster; paragraph, Antfly 1.33× faster; 350-word page, Upstream 1.20× faster; Wikipedia paragraph, Antfly 1.23× faster; entities + classes + relations, Antfly 1.48× faster; 8 paragraphs per request, Antfly 1.10× faster; 32 paragraphs per request, Antfly 1.20× faster. memory sentence, Antfly 1.11× smaller memory footprint (Antfly 2.06 GB vs upstream 2.29 GB); paragraph, Antfly 1.10× smaller memory footprint (Antfly 2.08 GB vs upstream 2.29 GB); 350-word page, Upstream 1.02× smaller memory footprint (Antfly 2.09 GB vs upstream 2.05 GB); Wikipedia paragraph, Antfly 1.10× smaller memory footprint (Antfly 2.08 GB vs upstream 2.29 GB); entities + classes + relations, Antfly 1.17× smaller memory footprint (Antfly 2.08 GB vs upstream 2.44 GB); 8 paragraphs per request, Antfly 1.38× smaller memory footprint (Antfly 2.32 GB vs upstream 3.19 GB); 32 paragraphs per request, Antfly 1.81× smaller memory footprint (Antfly 2.32 GB vs upstream 4.20 GB). EmbeddingGemma 2 (vs sentence-transformers 6.2.0): Speed 7/7, memory 7/7 complete. speed sentence, Antfly 1.75× faster; 32 sentences per request, Upstream 2.17× faster; paragraph, Antfly 1.49× faster; 32 paragraphs per request, Upstream 1.22× faster; 8 pages per request, Upstream 1.52× faster; ~2K-token document, Upstream 1.16× faster; ~7K-token document, Antfly 1.38× faster. memory sentence, Antfly 1.46× smaller memory footprint (Antfly 2.65 GB vs upstream 3.87 GB); 32 sentences per request, Antfly 1.44× smaller memory footprint (Antfly 2.70 GB vs upstream 3.87 GB); paragraph, Antfly 1.43× smaller memory footprint (Antfly 2.71 GB vs upstream 3.87 GB); 32 paragraphs per request, Antfly 2.11× smaller memory footprint (Antfly 2.78 GB vs upstream 5.86 GB); 8 pages per request, Antfly 1.39× smaller memory footprint (Antfly 2.77 GB vs upstream 3.84 GB); ~2K-token document, Antfly 1.44× smaller memory footprint (Antfly 2.79 GB vs upstream 4.00 GB); ~7K-token document, Antfly 2.85× smaller memory footprint (Antfly 3.06 GB vs upstream 8.74 GB). GLiNER2.5-Decide (vs Fastino gliner2 2.1.0): Speed 3/3, memory 3/3 complete. speed 1 review per request, Upstream 3.14× faster; 8 per request, Upstream 2.49× faster; 32 per request, Upstream 2.48× faster. memory 1 review per request, Antfly 1.02× smaller memory footprint (Antfly 4.21 GB vs upstream 4.28 GB); 8 per request, Upstream 1.47× smaller memory footprint (Antfly 5.24 GB vs upstream 3.56 GB); 32 per request, Upstream 1.18× smaller memory footprint (Antfly 5.25 GB vs upstream 4.46 GB). Gemma 4 E4B (vs llama.cpp b10809): Speed 5/5, memory 5/5 complete. speed short prompt, 256 tokens out, Antfly 1.61× faster; 350-word page, 128 out, Antfly 1.45× faster; ~2K-token prompt, 64 out, Antfly 1.10× faster; image, 64 out, Upstream 1.28× faster; 5–15 s audio, 64 out, Antfly 1.29× faster. memory short prompt, 256 tokens out, Upstream 1.09× smaller memory footprint (Antfly 7.23 GB vs upstream 6.66 GB); 350-word page, 128 out, Upstream 1.13× smaller memory footprint (Antfly 7.66 GB vs upstream 6.75 GB); ~2K-token prompt, 64 out, Upstream 1.20× smaller memory footprint (Antfly 8.11 GB vs upstream 6.75 GB); image, 64 out, Upstream 1.30× smaller memory footprint (Antfly 8.78 GB vs upstream 6.77 GB); 5–15 s audio, 64 out, Upstream 1.11× smaller memory footprint (Antfly 7.43 GB vs upstream 6.70 GB). Same prompt token count on all 46 inputs; generated text matches llama.cpp on none. Not identical outputs.. Whisper small (vs Transformers 5.23.0): Speed 3/3, memory 3/3 complete. speed 5.9 s clip, Upstream 5.40× faster; 15.1 s clip, Upstream 4.79× faster; 29.4 s clip, Upstream 4.45× faster. memory 5.9 s clip, Upstream 1.14× smaller memory footprint (Antfly 3.34 GB vs upstream 2.95 GB); 15.1 s clip, Upstream 1.15× smaller memory footprint (Antfly 3.46 GB vs upstream 3.01 GB); 29.4 s clip, Upstream 1.20× smaller memory footprint (Antfly 3.62 GB vs upstream 3.02 GB). LightOnOCR-3 0.8B (vs Transformers 5.23.0): Speed 1/1, memory 1/1 complete. speed one page, Antfly 1.07× faster. memory one page, Antfly 1.78× smaller memory footprint (Antfly 2.76 GB vs upstream 4.90 GB)
- Speed: upstream time ÷ Antfly time
- Peak memory: upstream ÷ Antfly
- Hollow: upstream better on that workload
Speed by model · Metal
- Model
- GLiNER2.5 base
- Compared against
- Fastino gliner2 2.1.0
- Typical workload
- 1.23× faster
- Range
- 1.20× slower to 1.83× faster
- Model
- EmbeddingGemma 2
- Compared against
- sentence-transformers 6.2.0
- Typical workload
- 1.16× slower
- Range
- 2.17× slower to 1.75× faster
- Model
- GLiNER2.5-Decide
- Compared against
- Fastino gliner2 2.1.0
- Typical workload
- 2.49× slower
- Range
- 3.14× to 2.48× slower
- Model
- Gemma 4 E4B
- Compared against
- llama.cpp b10809
- Typical workload
- 1.29× faster
- Range
- 1.28× slower to 1.61× faster
- Model
- Whisper small
- Compared against
- Transformers 5.23.0
- Typical workload
- 4.79× slower
- Range
- 5.40× to 4.45× slower
- Model
- LightOnOCR-3 0.8B
- Compared against
- Transformers 5.23.0
- Typical workload
- 1.07× faster
- Range
- One workload
Details
Antfly is slower on CUDA for EmbeddingGemma 2 with batches and long documents (1.4–4.6×); on Metal for Whisper small (4.5–5.4×), GLiNER2.5-Decide (2.5–3.1×), EmbeddingGemma 2 (up to 2.2×), Gemma 4 E4B on image input (1.3×) and GLiNER2.5 base on the 350-word page (1.2×). On Metal it uses 1.1–1.5× more memory for Gemma 4 E4B, Whisper small and batched GLiNER2.5-Decide.
These are selected workloads.
Tested versions
- Antfly
- v0.3.1
- Fastino gliner2
- 2.1.0
- sentence-transformers
- 6.2.0 · PyTorch 2.15.0 · CUDA and MPS · BF16
- llama.cpp
- b10809 · llama-server · CUDA and Metal · same Q4_0 GGUF files
- Hugging Face Transformers
- 5.23.0 · PyTorch 2.15.0 · CUDA and MPS
Do the outputs match upstream? Same on CUDA and Metal unless noted
- Model
- GLiNER2.5 base
- Agreement with upstream
- 1,344/1,344 paired outputs identical.
- Model
- EmbeddingGemma 2
- Agreement with upstream
- Cosine ≥0.9997 on every timed vector. SciFact, 24 queries: recall@1 0.93 vs 0.89 on CUDA, 0.89 vs 0.89 on Metal.
- Model
- GLiNER2.5-Decide
- Agreement with upstream
- 960/960 paired outputs identical.
- Model
- Gemma 4 E4B
- Agreement with upstream
- Generated text identical on 0/46 inputs; audio word error rate 5.3% vs llama.cpp 4.8%. GSM8K: 5/12 correct vs 4/12. ARC-Challenge: 6/12 vs 3/12.
- Model
- Whisper small
- Agreement with upstream
- 42 of 568 words wrong on 32 LibriSpeech clips; Transformers 43.
- Model
- LightOnOCR-3 0.8B
- Agreement with upstream
- Both read the timed page, 22 words, without error; the text is identical.
Methodology
- Measured November 17, 2026 (UTC) on Antfly v0.3.1. CUDA: NVIDIA L4. Metal: Apple M4 Max, 36 GiB.
- Both sides get identical input bytes and model checkpoints. Timing is the median of three paired, alternating runs after warm-up. Antfly timings include local HTTP and JSON overhead; references run in-process, except Gemma 4, which uses HTTP on both sides.
- Memory is the median peak of three fresh processes. CUDA: host plus GPU memory. Metal: the macOS memory footprint plus the model file held in RAM, counted in full on both sides. Part of that file may already be in the footprint, so the real figures can be lower.
- LightOnOCR: Transformers ran with generation caching on (use_cache=True); the checkpoint’s own setting leaves it off. Antfly’s time excludes the caller’s image encoding; the Transformers time includes its image preparation.
- Precision: both sides run the same checkpoint and differ in three places. Whisper: Transformers FP32; Antfly uses FP16 operands with FP32 accumulation, for weights on CUDA and encoder attention on Metal. LightOnOCR: Transformers computes in BF16, Antfly mostly in FP32. GLiNER2.5-Decide on Metal: Antfly holds the encoder weights in FP16.
Retrieval quality
STaRK-Prime measures document retrieval for specific biomedical questions. Quality is measured by Recall@20: the fraction of expected answers returned in the first 20 results, averaged across queries. Retrieval method matters: Antfly’s hybrid search over relation-expanded text reaches 49.34%, versus 36.00% for dense retrieval.
Benchmark definitionSTaRK-Prime · hybrid over relation-expanded text
Recall@20 across 2,801 questions. Higher is better.
Recall@20 across 2,801 questions. Higher is better.. Antfly: Antfly, 49.34%. Neo4j: Neo4j, 48.40%. Weaviate: Weaviate, 48.19%. Milvus: Milvus, 46.65%. Elasticsearch: Elasticsearch, 43.91%. Postgres + pgvector: Postgres + pgvector, 30.43%
- Antfly 0.3
- Other systems
Details
Tested versions
- Antfly
- v0.3.0
- Chroma
- 1.0.0
- Elasticsearch
- 9.4.1
- Neo4j
- 5.26.29
- Postgres + pgvector
- 0.8.5 on PostgreSQL 16.14
- Weaviate
- 1.38.2
- Milvus
- 3.0.0
- Quickwit
- 0.9.0
STaRK-Prime Recall@20 · every database and method
- Database
- Antfly
- Dense · official vectors
- 36.00%
- Lexical · default text
- 33.47%
- Hybrid · default text
- 40.41%
- Lexical · relation-expanded
- 45.32%
- Hybrid · relation-expanded
- 49.34%
- Database
- Neo4j
- Dense · official vectors
- 34.42%
- Lexical · default text
- 31.64%
- Hybrid · default text
- 39.21%
- Lexical · relation-expanded
- 43.06%
- Hybrid · relation-expanded
- 48.40%
- Database
- Weaviate
- Dense · official vectors
- 35.34%
- Lexical · default text
- 31.77%
- Hybrid · default text
- 39.24%
- Lexical · relation-expanded
- 43.80%
- Hybrid · relation-expanded
- 48.19%
- Database
- Milvus
- Dense · official vectors
- 32.61%
- Lexical · default text
- 31.76%
- Hybrid · default text
- 37.51%
- Lexical · relation-expanded
- 43.49%
- Hybrid · relation-expanded
- 46.65%
- Database
- Elasticsearch
- Dense · official vectors
- 35.78%
- Lexical · default text
- 31.58%
- Hybrid · default text
- 32.66%
- Lexical · relation-expanded
- 43.06%
- Hybrid · relation-expanded
- 43.91%
- Database
- Quickwit
- Dense · official vectors
- Unsupported
- Lexical · default text
- 30.85%
- Hybrid · default text
- Unsupported
- Lexical · relation-expanded
- 42.65%
- Hybrid · relation-expanded
- Unsupported
- Database
- Postgres + pgvector
- Dense · official vectors
- 35.55%
- Lexical · default text
- 3.07%
- Hybrid · default text
- 29.35%
- Lexical · relation-expanded
- 3.60%
- Hybrid · relation-expanded
- 30.43%
- Database
- Chroma
- Dense · official vectors
- 34.94%
- Lexical · default text
- Unsupported
- Hybrid · default text
- Unsupported
- Lexical · relation-expanded
- Unsupported
- Hybrid · relation-expanded
- Unsupported
| Database | Dense · official vectors | Lexical · default text | Hybrid · default text | Lexical · relation-expanded | Hybrid · relation-expanded |
|---|---|---|---|---|---|
| Antfly | 36.00% | 33.47% | 40.41% | 45.32% | 49.34% |
| Neo4j | 34.42% | 31.64% | 39.21% | 43.06% | 48.40% |
| Weaviate | 35.34% | 31.77% | 39.24% | 43.80% | 48.19% |
| Milvus | 32.61% | 31.76% | 37.51% | 43.49% | 46.65% |
| Elasticsearch | 35.78% | 31.58% | 32.66% | 43.06% | 43.91% |
| Quickwit | Unsupported | 30.85% | Unsupported | 42.65% | Unsupported |
| Postgres + pgvector | 35.55% | 3.07% | 29.35% | 3.60% | 30.43% |
| Chroma | 34.94% | Unsupported | Unsupported | Unsupported | Unsupported |
Methodology
- Apple M4 Max, Docker Desktop Linux arm64 VM (10 CPUs, 19.5 GiB), measured Nov 13, 2026.
- 129,375 candidate documents and 2,801 questions; every question counts in every score. All systems use the official text-embedding-ada-002 vectors. “Relation-expanded text” writes each record’s relationships into its text before indexing. Hybrid search merges lexical and vector results with reciprocal rank fusion.
- Antfly’s vector search ran at maximum search effort, which scans every vector: its dense score equals exact search, at 508–595 ms per dense or hybrid query. The other databases used approximate HNSW search with ef=200.
Full-text search
799 MIRACL English queries · 1 client · 250K–3M passages.
- Antfly v0.3
- Elasticsearch
- Quickwit
Query p95 (ms)
1 client · lower is better
Peak memory demand (MB)
anon + shmem · lower is better
Details
Figures are medians of three fresh indexes per engine and corpus size (storage is the mean); whiskers show the min–max. Peak memory excludes reclaimable file cache.
Antfly Recall@100 falls from 0.917 at 1M to 0.813 at 2M and 0.751 at 3M, and varies across fresh indexes (0.789–0.813 at 2M; 0.725–0.755 at 3M). Self-text checks missed 1–4 sampled relevant passages in Antfly and none in the other two engines. The cause is unresolved.
Tested versions
- Antfly
- v0.3.0
- Elasticsearch
- 9.5.2
- Quickwit
- 0.9.0
All engines and corpus sizes
- Engine
- Antfly
- Passages
- 250,000
- p95 (ms)
- 3.79
- nDCG@10
- 0.578
- Recall@100
- 0.940
- Peak memory (MB)
- 927.6
- Storage (MB)
- 193.9
- Engine
- Elasticsearch
- Passages
- 250,000
- p95 (ms)
- 3.28
- nDCG@10
- 0.552
- Recall@100
- 0.930
- Peak memory (MB)
- 1,814.3
- Storage (MB)
- 123.3
- Engine
- Quickwit
- Passages
- 250,000
- p95 (ms)
- 24.38
- nDCG@10
- 0.557
- Recall@100
- 0.942
- Peak memory (MB)
- 427.2
- Storage (MB)
- 284.9
- Engine
- Antfly
- Passages
- 1,000,000
- p95 (ms)
- 10.57
- nDCG@10
- 0.531
- Recall@100
- 0.917
- Peak memory (MB)
- 1,882.0
- Storage (MB)
- 746.0
- Engine
- Elasticsearch
- Passages
- 1,000,000
- p95 (ms)
- 4.75
- nDCG@10
- 0.493
- Recall@100
- 0.892
- Peak memory (MB)
- 1,941.1
- Storage (MB)
- 477.4
- Engine
- Quickwit
- Passages
- 1,000,000
- p95 (ms)
- 25.36
- nDCG@10
- 0.499
- Recall@100
- 0.907
- Peak memory (MB)
- 978.2
- Storage (MB)
- 714.2
- Engine
- Antfly
- Passages
- 2,000,000
- p95 (ms)
- 19.81
- nDCG@10
- 0.433
- Recall@100
- 0.813
- Peak memory (MB)
- 2,755.6
- Storage (MB)
- 1,649.0
- Engine
- Elasticsearch
- Passages
- 2,000,000
- p95 (ms)
- 6.49
- nDCG@10
- 0.453
- Recall@100
- 0.866
- Peak memory (MB)
- 1,917.4
- Storage (MB)
- 947.0
- Engine
- Quickwit
- Passages
- 2,000,000
- p95 (ms)
- 25.70
- nDCG@10
- 0.459
- Recall@100
- 0.880
- Peak memory (MB)
- 1,323.3
- Storage (MB)
- 1,422.2
- Engine
- Antfly
- Passages
- 3,000,000
- p95 (ms)
- 27.12
- nDCG@10
- 0.385
- Recall@100
- 0.751
- Peak memory (MB)
- 3,530.8
- Storage (MB)
- 2,201.6
- Engine
- Elasticsearch
- Passages
- 3,000,000
- p95 (ms)
- 9.43
- nDCG@10
- 0.425
- Recall@100
- 0.849
- Peak memory (MB)
- 1,973.7
- Storage (MB)
- 1,416.2
- Engine
- Quickwit
- Passages
- 3,000,000
- p95 (ms)
- 25.93
- nDCG@10
- 0.429
- Recall@100
- 0.858
- Peak memory (MB)
- 1,569.8
- Storage (MB)
- 2,003.0
Methodology
- Antfly, Elasticsearch 9.5.2 and Quickwit 0.9.0 in one run on a Google Cloud c4a-standard-16 VM (Google Axion ARM64, 16 vCPUs, 62.7 GiB), measured Nov 13, 2026.
- Every system indexes the same passages, and each smaller corpus is contained in the larger ones. Each run sends ten warm-up queries, then all 799 queries one at a time in three passes; the last pass is reported.
- Antfly runs as a single shard. After indexing, Elasticsearch is force-merged into one segment; Antfly gets no equivalent explicit compaction step, though it may still compact on its own. All three engines store the passage text.
Vector search
VectorDBBench on 1M 768-dimensional vectors, with every system tuned to about 95% recall: query latency, throughput, memory and index build time. 50K and filtered-search results are in Details.
Serial query p95
One client. Lower is better.
One client. Lower is better.. Antfly: Antfly, 9.633 ms. Chroma: Chroma, 6.133 ms. Elasticsearch: Elasticsearch, 3.133 ms. Postgres + pgvector: Postgres + pgvector, 10.200 ms. Weaviate: Weaviate, 5.200 ms. Milvus: Milvus, 1.900 ms
- Antfly 0.3
- Other systems
Peak throughput
Best rate across concurrency levels. Higher is better.
Best rate across concurrency levels. Higher is better.. Antfly: Antfly, 920/s. Chroma: Chroma, 242/s. Elasticsearch: Elasticsearch, 3,091/s. Postgres + pgvector: Postgres + pgvector, 1,144/s. Weaviate: Weaviate, 1,348/s. Milvus: Milvus, 3,379/s
- Antfly 0.3
- Other systems
Peak memory
Lower is better.
Lower is better.. Antfly: Antfly, 4,627 · 10,448 MB. Chroma: Chroma, 3,698 · 7,334 MB. Elasticsearch: Elasticsearch, 2,058 · 11,136 MB. Postgres + pgvector: Postgres + pgvector, 2,320 · 12,004 MB. Weaviate: Weaviate, 7,336 · 12,264 MB. Milvus: Milvus, 12,179 · 15,492 MB
- Memory it can’t release
- Including page cache
- Antfly 0.3
- Other systems
Index build time
Insert 1M vectors until the index reports all of them. Lower is better.
Insert 1M vectors until the index reports all of them. Lower is better.. Antfly: Antfly, 284 s. Chroma: Chroma, 837 s. Elasticsearch: Elasticsearch, 480 s. Postgres + pgvector: Postgres + pgvector, 812 s. Weaviate: Weaviate, 678 s. Milvus: Milvus, 586 s
- Antfly 0.3
- Other systems
Details
Antfly results are means of three fresh indexes.
Tested versions
- Chroma
- 1.5.9
- Elasticsearch
- 9.5.2
- Postgres + pgvector
- 0.8.6 (PostgreSQL 16)
- Weaviate
- 1.39.2
- Milvus
- 2.6.22
- Antfly
- v0.3.0
1M vectors, 768 dimensions
- Engine
- Antfly
- Recall
- 0.9627
- Peak QPS
- 920.1
- p95 (ms)
- 9.633
- Memory excl. page cache (MB)
- 4,627.0
- Storage (MB)
- 5,969.5
- Index build (s)
- 284
- Engine
- Chroma 1.5.9
- Recall
- 0.9562
- Peak QPS
- 241.6
- p95 (ms)
- 6.133
- Memory excl. page cache (MB)
- 3,697.7
- Storage (MB)
- 3,459.7
- Index build (s)
- 837
- Engine
- Elasticsearch 9.5.2
- Recall
- 0.9602
- Peak QPS
- 3,090.9
- p95 (ms)
- 3.133
- Memory excl. page cache (MB)
- 2,057.5
- Storage (MB)
- 3,148.2 (provisional)
- Index build (s)
- 480
- Engine
- Milvus 2.6.22
- Recall
- 0.9568
- Peak QPS
- 3,379.2
- p95 (ms)
- 1.900
- Memory excl. page cache (MB)
- 12,178.9
- Storage (MB)
- 25,413.7
- Index build (s)
- 586
- Engine
- pgvector 0.8.6-pg16
- Recall
- 0.9656
- Peak QPS
- 1,144.0
- p95 (ms)
- 10.200
- Memory excl. page cache (MB)
- 2,320.3
- Storage (MB)
- 9,329.0
- Index build (s)
- 812
- Engine
- Weaviate 1.39.2
- Recall
- 0.9563
- Peak QPS
- 1,348.5
- p95 (ms)
- 5.200
- Memory excl. page cache (MB)
- 7,336.5
- Storage (MB)
- 3,947.5
- Index build (s)
- 678
50K vectors, 1,536 dimensions
- Engine
- Antfly
- Recall
- 0.9621
- Peak QPS
- 1,071.9
- p95 (ms)
- 5.633
- Memory excl. page cache (MB)
- 1,576.4
- Storage (MB)
- 373.1
- Engine
- Chroma 1.5.9
- Recall
- 0.9869
- Peak QPS
- 182.0
- p95 (ms)
- 7.567
- Memory excl. page cache (MB)
- 461.0
- Storage (MB)
- 332.7
- Engine
- Elasticsearch 9.5.2
- Recall
- 0.9526
- Peak QPS
- 4,605.3
- p95 (ms)
- 1.800
- Memory excl. page cache (MB)
- 1,938.3
- Storage (MB)
- 312.1 (provisional)
- Engine
- Milvus 2.6.22
- Recall
- 0.9629
- Peak QPS
- 5,443.9
- p95 (ms)
- 1.400
- Memory excl. page cache (MB)
- 2,785.4
- Storage (MB)
- 2,393.4
- Engine
- pgvector 0.8.6-pg16
- Recall
- 0.9690
- Peak QPS
- 2,889.1
- p95 (ms)
- 4.067
- Memory excl. page cache (MB)
- 2,303.7
- Storage (MB)
- 1,524.0
- Engine
- Weaviate 1.39.2
- Recall
- 0.9547
- Peak QPS
- 1,037.4
- p95 (ms)
- 5.833
- Memory excl. page cache (MB)
- 976.5
- Storage (MB)
- 391.4
Filtered search, 1% tenant filter
- Engine
- Antfly
- Recall@100
- 1.0000
- Peak QPS
- 616.7
- Peak p99 (ms)
- 79.97
- Memory excl. page cache (MB)
- 1,029.4
- Storage (MB)
- 373.6
- Engine
- Elasticsearch 9.5.2
- Recall@100
- 1.0000
- Peak QPS
- 5,965.9
- Peak p99 (ms)
- 11.78
- Memory excl. page cache (MB)
- 1,916.0
- Storage (MB)
- 312.3 (provisional)
- Engine
- Weaviate 1.39.2
- Recall@100
- 1.0000
- Peak QPS
- 1,170.9
- Peak p99 (ms)
- 10.97
- Memory excl. page cache (MB)
- 980.1
- Storage (MB)
- 392.8
- Engine
- pgvector 0.8.6-pg16
- Recall@100
- 0.9573
- Peak QPS
- 163.8
- Peak p99 (ms)
- 218.15
- Memory excl. page cache (MB)
- 2,303.8
- Storage (MB)
- 1,524.0
- Engine
- Chroma 1.5.9
- Recall@100
- 0.9999
- Peak QPS
- 112.0
- Peak p99 (ms)
- 411.46
- Memory excl. page cache (MB)
- 495.9
- Storage (MB)
- 336.7
Methodology
- Apple M4 Max, Docker Desktop Linux arm64 VM (10 CPUs, 19.5 GiB), measured Nov 13, 2026.
- Search settings target at least 95% recall. Antfly’s 50K search effort was raised from 0.425 to 0.45 after a 0.949 result, so those queries were seen during tuning. Filtered search restricts each query to a tenant holding 1% of the data.