AntflyAntfly
Back to Research

Building a Distributed Search Engine in Pure Go

How we built the Antfly engine in Go: Multi-Raft consensus, SIMD-accelerated vectors, and local ML inference, all in one static binary.

AJ Roetker
AJ Roetker
CTO
Godistributed-systemssearchraftSIMDMLvector-db

Multi-Raft consensus, SIMD-accelerated vectors, and local ML inference, all without leaving the Go toolchain.

Hybrid search usually means standing up a vector store, a search engine, a graph database, an embedding service, and a reranker, then keeping the five of them in sync. Antfly does all of it in one Go binary.

It does full-text search (BM25), dense vector similarity, SPLADE sparse vectors, and graph traversal, and it runs its own ML inference for embeddings, reranking, and chunking. You go from raw documents to working retrieval without assembling a stack first.

This post is about the Go-specific decisions behind it: why we run many small Raft groups instead of one big one, how we get SIMD vector math in pure Go, and what it took to make local inference fast enough to sit in a query path.

The shape of it#

Run antfly standalone and you get a full node in a single process: storage, indexing, search, ML inference, an MCP server, and a dashboard. Add nodes and the cluster splits shards and replicates on its own.

Consensus is etcd's Raft library (go.etcd.io/raft/v3), but not as one monolithic group. Antfly runs Multi-Raft: a single metadata group owns table schemas, the shard map, and cluster topology, and every data shard gets its own independent Raft group with its own log, leader, and snapshots. A 16-shard table across 3 replicas is 16 small Raft groups, not one big one. The payoff is isolation. A hot shard doesn't stall consensus on the cold ones, and splitting a shard is a local operation instead of a global stop-the-world.

Underneath, Pebble (the LSM storage engine from CockroachDB) holds both the data and the Raft log. One storage engine, one set of compaction knobs to understand.

SIMD vectors, in pure Go#

Vector similarity is the hot loop in any embedding search. Cosine similarity across millions of float32 vectors has to be fast, and historically your Go options were CGo (which fights the goroutine scheduler) or hand-written assembly (which nobody wants to own).

Go 1.26 changed the math. With GOEXPERIMENT=simd, Go programs get native access to SIMD instructions, and Antfly drives them through go-highway, a portable SIMD library we maintain, inspired by Google's Highway. You write the kernel once and it dispatches to AVX2, AVX-512, or ARM NEON at runtime, with a plain-Go fallback when none of those are available. No CGo, no per-platform assembly to babysit.

// go-highway: write the kernel once, dispatch at runtime
func cosineSimilarity(a, b []float32) float32 {
    va, vb := hwy.Load(a), hwy.Load(b)
    dot := hwy.ReduceSum(hwy.Mul(va, vb))
    na := hwy.ReduceSum(hwy.Mul(va, va))
    nb := hwy.ReduceSum(hwy.Mul(vb, vb))
    return dot / float32(math.Sqrt(float64(na))*math.Sqrt(float64(nb)))
}

This is CPU-first by design. The dense vector index, the SPLADE sparse vectors, and the BM25 full-text index (built on a fork of Bleve) all run on the machine you already have. You don't need a GPU to get fast hybrid search.

Inference next to the data#

Antfly ships Antfly Inference, a local engine for embeddings, reranking, chunking, entity extraction, and OCR. It runs models in-process on top of a gomlx fork and an ONNX runtime, so a write can turn into a dense embedding, a SPLADE sparse vector, and graph edges without a network hop.

Keeping inference local matters for three plain reasons. Latency: a query that embeds itself before it searches doesn't pay a round trip to an external API first. Cost: local inference spends CPU you're already paying for instead of per-request API charges. Privacy: the documents never leave your VPC, which for some teams is the whole conversation.

Local is the default, not the cage. When you'd rather call out, Antfly Inference takes providers like Ollama, OpenAI, Bedrock, Vertex, Gemini, Cohere, and OpenRouter, so you can mix a hosted model into the same pipeline.

Hybrid search is one request#

The point of putting all of this in one engine is that an application asks one database for the result it wants and gets back a single fused, ranked list.

# BM25 + dense + SPLADE + graph traversal, fused and reranked
curl -X POST http://localhost:8080/db/v1/tables/docs/query \
  -H 'Content-Type: application/json' \
  -d '{
    "full_text_search": {
      "match": { "field": "body", "text": "raft consensus in distributed databases" }
    },
    "semantic_search": "raft consensus in distributed databases",
    "indexes": ["body_dense", "body_sparse"],
    "merge_config": {
      "strategy": "rsf",
      "weights": { "full_text": 0.4, "body_dense": 0.8, "body_sparse": 0.6 }
    },
    "graph_searches": {
      "related_docs": {
        "type": "traverse",
        "index_name": "citations",
        "start_nodes": { "result_ref": "$full_text_results", "limit": 5 },
        "params": { "direction": "out", "max_depth": 1 }
      }
    },
    "reranker": {
      "provider": "antfly",
      "model": "cross-encoder/ms-marco-MiniLM-L-6-v2",
      "field": "body"
    },
    "limit": 10
  }'

One request fans out, in parallel goroutines, to BM25, the dense index, the sparse index, and a graph traversal, fuses the results, and reranks the top of the list with a local cross-encoder. Goroutines are what make that cheap: each branch of the query is its own lightweight routine, and Go schedules thousands of them without thread-pool tuning or async bookkeeping.

And it's quick. On the 50K VectorDBBench set, on an M4 Max, the Go engine indexes the corpus in about 71 seconds and answers queries at roughly 10 to 12 ms p95, at 0.9975 recall.

Getting it right#

Distributed systems fail in the gaps between normal operations, so we specify the hard parts before we trust them. Antfly has TLA+ specifications for its core protocols, checked with the TLC model checker:

SpecWhat it pins down
AntflyTransactionDistributed transaction correctness
occ-2pcOptimistic concurrency with two-phase commit
AntflySnapshotTransferRaft snapshot transfer safety
AntflyShardSplitShard split coordination under concurrent writes

The transaction spec also has trace validation: the implementation emits execution traces that we replay against the model to confirm the running code respects the invariants we wrote down. Below that sits an end-to-end vector-search benchmark harness that measures insert and query paths with p50/p95/p99 latency and recall, so a change to a kernel or an index shows up as a number, not a vibe.

One binary, many ways in#

A single antfly binary exposes more surface than you might expect:

InterfaceWhat it is
REST APIHTTP/JSON for search, indexing, and retrieval
MCP serverModel Context Protocol, so an LLM can call the database as a tool
A2AAgent-to-Agent protocol for inter-agent communication
Go SDKgithub.com/antflydb/antfly/go/pkg/sdk
TypeScript SDK@antfly/sdk, with React components
Python SDKthe antfly package
Operatora Kubernetes operator for declarative clusters

There's also a small ecosystem of tools around the engine: pgaf, a Postgres extension (written in Rust) that adds Antfly-backed search to an existing database; docsaf for pulling text out of messy documents; and evalaf for measuring retrieval quality. The core server is Elastic License v2 (self-host it, modify it, build on it; just don't resell Antfly itself as a managed service), and the surrounding tools are Apache 2.0.

Getting started#

# With Docker, inference models baked in
docker run -p 8080:8080 ghcr.io/antflydb/antfly:omni

# Or, with the binary installed
antfly standalone

The Antfarm dashboard comes up on :8080 with playgrounds for search, retrieval, graphs, embeddings, and reranking, which is the fastest way to poke at the API before you write any code.

That one static binary is the whole case for Go at this layer. No virtualenv, no pip tree, no CUDA driver to line up: the search, inference, and agent interfaces that sit under an application, called hundreds of times a session, all run in one process you can drop on a box, air-gapped or at the edge. Go's low-pause garbage collector keeps that process from stalling in the middle of a retrieval. For infrastructure, that is a good trade.

[Update] (May 2026)#

Since this post went out, we've rebuilt the engine from the runtime up in Zig, for tighter control over threads, memory, and dependencies, and a good bit more speed. If you want that story, it's here: A Search-and-Inference Database from Scratch in Pure Zig.

Written by AJ Roetker. Questions or feedback? Find us on Discord or open an issue on GitHub.