Tune Hybrid Search

The Result#

One query finds documents that contain your words and documents that mean the same thing, fuses the two lists, reorders them with a cross-encoder, and drops the weak tail. Each stage is independent, and the steps below add them one at a time:

curl -X POST http://127.0.0.1:8080/db/v1/query \
  -H "Content-Type: application/json" \
  -d '{
    "table": "wikipedia",
    "full_text_search": {"query": "body:Einstein"},
    "semantic_search": "theory of relativity and physics",
    "indexes": ["title_body"],
    "fields": ["title", "url"],
    "merge_config": {"strategy": "rsf"},
    "limit": 10,
    "reranker": {
      "provider": "antfly",
      "model": "ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF:gguf:Q8_0",
      "field": "body",
      "candidate_count": 50
    },
    "pruner": {"min_score_ratio": 0.5, "max_score_gap_percent": 40}
  }'

Before You Start#

The Wikipedia table from the Quickstart, loaded and indexed:

antfly query --table wikipedia --full-text-search 'body:Einstein' --fields title --limit 1

Build It#

1. Run Both Searches in One Query#

full_text_search finds documents that contain the words. semantic_search finds documents that mean the same thing. Put both in one request and Antfly runs them in parallel and fuses the ranked lists:

antfly query --table wikipedia \
  --full-text-search 'body:Einstein' \
  --semantic-search "theory of relativity and physics" \
  --indexes "title_body" \
  --fields "title,url" \
  --limit 10

Keyword search misses synonyms and paraphrases: "automobile" never finds "cars". Semantic search misses exact strings a keyword index catches instantly: an error code, a part number, a name spelled one way. Running both covers each one's blind spot.

2. Choose How the Lists Fuse#

merge_config controls the fusion. The default is Reciprocal Rank Fusion:

score = sum over lists of 1 / (rank_constant + rank)

rank_constant defaults to 60. A document ranked first in both lists scores 1/61 + 1/61; first in one and tenth in the other scores 1/61 + 1/70. Only rank positions matter, so RRF is robust across scoring scales with no tuning. The cost is that it compresses every score into a narrow band, which leaves a pruner little to work with.

Relative Score Fusion keeps score magnitude. It min-max normalizes each list within a window (window_size, defaulting to limit) and combines them, producing a wider spread:

curl -X POST http://127.0.0.1:8080/db/v1/query \
  -H "Content-Type: application/json" \
  -d '{
    "table": "wikipedia",
    "full_text_search": {"query": "body:Einstein"},
    "semantic_search": "theory of relativity",
    "indexes": ["title_body"],
    "merge_config": {"strategy": "rsf"},
    "limit": 10
  }'

Both strategies accept weights, keyed by index name with full_text for the keyword side. Unlisted indexes weigh 1.0:

"merge_config": {
  "strategy": "rrf",
  "weights": {"full_text": 0.3, "title_body": 1.0}
}

Lean the weights toward full_text when exact matches must win (catalog search, log search). Lean toward the embeddings index when users describe what they want in their own words.

The CLI always fuses with RRF; use the REST API or an SDK for merge_config.

3. Rerank the Candidates#

Fusion gets the right documents into the pool. A cross-encoder reranker reads each query-document pair in full and rescores it, which orders the pool far more accurately than either retrieval score:

antfly query --table wikipedia \
  --full-text-search 'body:Einstein' \
  --semantic-search "theory of relativity and physics" \
  --indexes "title_body" \
  --fields "title,url" \
  --limit 10 \
  --reranker '{
    "provider": "antfly",
    "model": "ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF:gguf:Q8_0",
    "field": "body",
    "candidate_count": 50
  }'

field names the document text the reranker scores; use template instead when the text spans several fields. candidate_count is the scoring window and limit is what comes back: Antfly retrieves a bounded window from each shard, keeps the globally best candidate_count hits, makes one batched reranker call at the coordinator, and applies offset and limit after that. Set candidate_count to at least offset + limit, usually 50 to 100. The window is capped at 1,000 (200 on Vertex), and an oversized request returns 422 reranker_candidate_limit_exceeded before any work starts. hits.total still reports the retrieval match count. The legacy top_n field remains as a deprecated limit override.

The reranker providers that execute are antfly, cohere, and vertex. With antfly, model may be omitted and the runtime picks a model from its reranker directory; set it when more than one reranker is installed. A larger window improves recall and costs latency, and on a hosted provider, money.

4. Prune the Tail#

Pruning runs once at the coordinator after global fusion and reranking, before offset and limit select the page, so standalone and distributed deployments share one score domain. Instead of returning a fixed count, it returns only what clears a quality bar. The strategies combine in one pruner object:

antfly query --table wikipedia \
  --full-text-search 'body:Einstein' \
  --semantic-search "theory of relativity and physics" \
  --indexes "title_body" \
  --fields "title,url" \
  --limit 10 \
  --pruner '{"min_score_ratio": 0.5, "max_score_gap_percent": 30}'

min_score_ratio and max_score_gap_percent together are the usual pair: one sets a floor relative to the best result, the other stops at the elbow where relevance falls off. Pair pruning with RSF; RRF's compressed scores give the gap detector nothing to see.

Tradeoffs#

Which stages to turn on depends on how your users ask.

Start with the default and add stages when you can name what each one fixes. Every stage is optional and independent, and the query still returns results when a later stage has nothing to do.

Use Agent Skills#

Everything above is also encoded in the Antfly skill, so a coding agent can execute this guide for you:

npx skills add antflydb/antfly-skills

Then prompt it with the outcome, for example "Add hybrid search with RSF fusion, a reranker, and a pruner to my products table", and use this page to judge the result.

Next Steps#

• Build a Support Answer Agent: this pipeline feeding a generated answer
• Document Engine: how the planner combines full-text, vector, filters, and sort under one query
• Antfly Inference: swapping the embedding and reranking models