Overview#

Hybrid search combines two complementary retrieval methods:

• Full-text search (BM25) excels at exact keyword matching. It finds documents containing specific terms, phrases, and field-scoped queries. But it misses synonyms and paraphrases entirely -- searching for "automobile" won't find documents about "cars."

• Semantic search (vector similarity) understands meaning. It finds conceptually related documents even when they use different words. But it can miss exact matches that a keyword search would catch instantly.

AntflyDB runs both searches in parallel and fuses the results into a single ranked list. You get the precision of keyword matching and the recall of semantic understanding in one query.

Getting Started#

This guide assumes you have AntflyDB running with the Wikipedia dataset from the quickstart guide. If you haven't set that up yet, start there first.

Your First Hybrid Query#

A hybrid query uses both --full-text-search and --semantic-search together:

antfly query --table wikipedia \
  --full-text-search 'body:Einstein' \
  --semantic-search "theory of relativity and physics" \
  --indexes "title_body" \
  --fields "title,url" \
  --limit 10

The same query via the REST API:

curl -X POST http://localhost:8080/v1/query \
  -H "Content-Type: application/json" \
  -d '{
    "table": "wikipedia",
    "full_text_search": {"query": "body:Einstein"},
    "semantic_search": "theory of relativity and physics",
    "indexes": ["title_body"],
    "fields": ["title", "url"],
    "limit": 10
  }'

AntflyDB runs both searches simultaneously, then merges the results using Reciprocal Rank Fusion (the default strategy).

Merge Strategies#

When both full-text and semantic results come back, AntflyDB needs to combine them into a single ranked list. The merge_config.strategy field controls how this happens.

RRF (Reciprocal Rank Fusion) -- Default#

RRF scores each result based on its rank position across both result sets:

score = sum(1 / (k + rank))  for each source where the result appears

The constant k is set to 60. A result ranked #1 in both lists gets 1/61 + 1/61 = 0.0328. A result ranked #1 in one list and #10 in the other gets 1/61 + 1/70 = 0.0307.

RRF ignores raw scores entirely -- only rank positions matter. This makes it robust across different scoring scales without any tuning. It is the right choice for most use cases.

The tradeoff: because RRF compresses all scores into a narrow range, it can make it harder to distinguish high-relevance results from marginal ones when pruning.

RSF (Relative Score Fusion)#

RSF normalizes raw scores from each source using min-max normalization, then combines them:

normalized = (score - min) / (max - min)  within a sliding window

Because RSF preserves score magnitude information, it produces a wider spread of fused scores. This makes it a better fit when you plan to use pruning to filter out low-quality results.

curl -X POST http://localhost:8080/v1/query \
  -H "Content-Type: application/json" \
  -d '{
    "table": "wikipedia",
    "full_text_search": {"query": "body:Einstein"},
    "semantic_search": "theory of relativity",
    "indexes": ["title_body"],
    "merge_config": {"strategy": "rsf"},
    "limit": 10
  }'

RSF accepts window_size and per-source weights in merge_config. Unspecified sources use a weight of 1.0:

{
  "merge_config": {
    "strategy": "rsf",
    "window_size": 50,
    "weights": {"full_text": 0.5, "title_body": 1.0}
  }
}

Reranking#

After fusion, results are ranked but not perfectly ordered. A cross-encoder reranker can re-score each result by looking at the full query-document pair, producing significantly more accurate relevance scores.

antfly query --table wikipedia \
  --full-text-search 'body:Einstein' \
  --semantic-search "theory of relativity and physics" \
  --indexes "title_body" \
  --fields "title,url" \
  --limit 10 \
  --reranker '{
    "provider": "antfly",
    "field": "body",
    "candidate_count": 50
  }'

The field parameter tells the reranker which document field to score against the query. candidate_count controls the expensive scoring window, while the query's limit controls the number returned. When model is omitted, the antfly provider selects a model from the inference service's reranker directory; set it explicitly when more than one reranker is installed, for example mixedbread-ai/mxbai-rerank-base-v1.

The executable reranker providers are antfly, cohere, and vertex. Antfly retrieves the bounded window from each shard, retains the globally best candidate_count hits, and makes one batched reranker call at the coordinator. offset and limit are applied after reranking. The legacy reranker top_n field remains as a deprecated limit override and is validated against candidate_count. Reranking changes the returned order and page, while hits.total continues to report the underlying retrieval match count. The effective candidate window is capped at 1,000, including when it is derived from offset + limit. Vertex has a provider-specific ceiling of 200. Oversized requests return 422 reranker_candidate_limit_exceeded with the selected provider and its exact maximum before shard fan-out or provider work begins.

Pruning#

Pruning filters out low-quality results after global fusion (and after reranking, if configured), before offset and limit select the final page. It runs once at the coordinator, so standalone and distributed deployments use the same score domain. Instead of returning a fixed number of results, pruning returns only the results that meet a quality threshold.

This is especially valuable for RAG pipelines, where sending marginally relevant documents to an LLM adds noise and degrades answer quality.

Pruning Strategies#

You can combine multiple strategies in a single pruner configuration:

min_score_ratio -- Keep results scoring at least X% of the top result.

{"min_score_ratio": 0.5}

A value of 0.5 keeps results with at least half the score of the best result.

max_score_gap_percent -- Stop when the score drops too sharply from one result to the next.

{"max_score_gap_percent": 30}

This detects "elbows" in the score distribution. A value of 30 stops returning results when a score drops more than 30% from the previous result.

min_absolute_score -- Hard floor on score values.

{"min_absolute_score": 0.01}

std_dev_threshold -- Statistical outlier removal. Keeps results within N standard deviations below the mean.

{"std_dev_threshold": 1.5}

require_multi_index -- Only keep results that appear in both full-text and vector results.

{"require_multi_index": true}

Example with Pruning#

antfly query --table wikipedia \
  --full-text-search 'body:Einstein' \
  --semantic-search "theory of relativity and physics" \
  --indexes "title_body" \
  --fields "title,url" \
  --limit 50 \
  --pruner '{"min_score_ratio": 0.5, "max_score_gap_percent": 30}'

Putting It All Together#

Here is a complete hybrid search pipeline that combines all the pieces: full-text search, semantic search, merge strategy, reranking, and pruning.

curl -X POST http://localhost:8080/v1/query \
  -H "Content-Type: application/json" \
  -d '{
    "table": "wikipedia",
    "full_text_search": {"query": "body:Einstein"},
    "semantic_search": "theory of relativity and physics",
    "indexes": ["title_body"],
    "fields": ["title", "body", "url"],
    "merge_config": {"strategy": "rsf"},
    "limit": 10,
    "reranker": {
      "provider": "antfly",
      "field": "body",
      "candidate_count": 50
    },
    "pruner": {
      "min_score_ratio": 0.5,
      "max_score_gap_percent": 40
    }
  }'

This query:

1. Runs a BM25 keyword search for "Einstein" in the body field
2. Runs a semantic similarity search for "theory of relativity and physics"
3. Fuses results using RSF (preserving score magnitude)
4. Reranks the top 50 candidates with a cross-encoder model and keeps the best 10
5. Prunes those results below 50% of the top score or where the score drops sharply

Feeding Into RAG#

The same pipeline works with antfly agents retrieval to generate AI-powered answers from the search results:

antfly agents retrieval --table wikipedia \
  --full-text-search 'body:Einstein' \
  --semantic-search "What is the theory of relativity?" \
  --indexes "title_body" \
  --fields "title,body" \
  --limit 50 \
  --reranker '{
    "provider": "antfly",
    "field": "body"
  }' \
  --pruner '{"min_score_ratio": 0.6, "max_score_gap_percent": 40}' \
  --generator '{
    "provider": "ollama",
    "model": "gemma3:4b-it-qat",
    "url": "http://localhost:11434"
  }' \
  --system-prompt "Answer the question based on the provided context."

See the quickstart guide for more on setting up RAG with Ollama.

When to Use What#

Additional Resources#

• Getting Started -- Set up AntflyDB and load the Wikipedia dataset
• REST API Reference -- Full query API documentation
• Inference Guide -- Configure custom embedding and reranking models
• Configuration Reference -- Server and index configuration options