Pull the CLIP model#

antfly termite pull openai/clip-vit-base-patch32

Add an image index to your table#

Add a CLIP-based index to embed each article's Wikipedia thumbnail:

antfly index create --table wikipedia \
  --index thumbnail \
  --type embeddings \
  --template "{{media url=thumbnail_url}}" \
  --embedder '{"provider": "termite", "model": "openai/clip-vit-base-patch32"}' \
  --dimension 512

Adding an index to a table with existing data triggers background embedding generation. Articles without a thumbnail_url are skipped. Monitor progress with antfly index list --table wikipedia.

Search by visual content#

CLIP understands visual concepts, so you can describe what you're looking for in natural language:

antfly query --table wikipedia \
  --semantic-search "map of a country" \
  --indexes "thumbnail" \
  --fields "title,url,thumbnail_url" \
  --limit 5

Pull a generation model#

antfly termite pull hf:onnxruntime/Gemma-3-ONNX

Classify and generate#

Use the Gemma model to classify a query and generate an answer from search results:

antfly agents retrieval --table wikipedia \
  --semantic-search "What are the major events in Korean history?" \
  --indexes "title_body" \
  --fields "title,body" \
  --limit 5 \
  --reranker '{
      "provider": "termite",
      "model": "mixedbread-ai/mxbai-rerank-base-v1",
      "field": "body"
    }' \
  --pruner '{"min_score_ratio": 0.6, "max_score_gap_percent": 40}' \
  --generator '{
    "provider": "termite",
    "model": "onnxruntime/Gemma-3-ONNX"
  }' \
  --max-context-tokens 512 \
  --classify --reasoning --generate --followup

This command searches for semantically similar articles, reranks and prunes results for quality, classifies the query, generates an answer with reasoning, and suggests follow-up questions — all using the local Gemma model.

RAG with alternative providers#

The previous step uses Termite for local inference. You can also use cloud APIs or Ollama:

export GEMINI_API_KEY="your-api-key-here"

antfly agents retrieval --table wikipedia \
  --semantic-search "What are the major events in Korean history?" \
  --indexes "title_body" \
  --fields "title,body" \
  --limit 5 \
  --generator '{"provider": "gemini", "model": "gemini-2.5-flash", "api_key": "${GEMINI_API_KEY}"}' \
  --max-context-tokens 2048 \
  --classify --reasoning --generate --followup

Structured output#

Add --streaming=false to get structured JSON output with source references instead of streaming text:

antfly agents retrieval --table wikipedia \
  --semantic-search "Explain the theory of relativity" \
  --indexes "title_body" \
  --fields "title,body,url" \
  --limit 5 \
  --generator '{"provider": "gemini", "model": "gemini-2.5-flash", "api_key": "${GEMINI_API_KEY}"}' \
  --streaming=false --generate

Evaluation#

Add --eval to score the quality of retrieval and generation:

antfly agents retrieval --table wikipedia \
  --semantic-search "What are the major events in Korean history?" \
  --indexes "title_body" \
  --fields "title,body" \
  --limit 5 \
  --generator '{"provider": "gemini", "model": "gemini-2.5-flash", "api_key": "${GEMINI_API_KEY}"}' \
  --generate \
  --eval '{
    "evaluators": ["faithfulness", "relevance"],
    "judge": {"provider": "gemini", "model": "gemini-3.0-flash", "api_key": "${GEMINI_API_KEY}"}
  }'

Available evaluators:

• Retrieval metrics (require ground_truth.relevant_ids): recall, precision, ndcg, mrr, map
• LLM-as-judge metrics (require judge config): relevance, faithfulness, completeness, coherence, safety, helpfulness, correctness, citation_quality