AntflyAntfly

Quickstart

Install Antfly, load data, and run your first queries

Beginner45–90 min
getting-startedsetupquickstart
Prerequisites
  • curl and jq (for sample data)

By the end of this page you have Antfly running on your machine with 10,000 Wikipedia articles loaded, and you have searched them by keyword, by meaning, and by what their pictures show, then asked a local model to answer a question from them, all on your own machine with no hosted service involved.

Allow 45 to 90 minutes on a laptop. The first full-text check takes seconds; model downloads and background indexing take most of the time, and slower hardware may need longer timeouts.

Install Antfly#

brew install antflydb/taps/antfly

# Verify
antfly --version

Start Antfly#

Start Antfly in standalone mode:

antfly standalone --inference-host-budget-mb 8192 2>&1 | tee "antfly.log"

If you installed with Homebrew, you can run Antfly as a service instead:

brew services start antfly

Standalone mode runs metadata, data, and Antfly Inference in one process. The 8 GiB inference budget is what lets the all-local RAG step at the end run the embedding, reranking, and Gemma models in one request; drop the flag if you plan to use a hosted generator. The web dashboard is at http://127.0.0.1:8080 once it is up. Every command on this page uses the numeric loopback address because it matches standalone's default bind on systems where localhost resolves to IPv6 first.

Verify the basic data path#

One document in, one document out, before anything expensive:

curl -fsS -X POST http://127.0.0.1:8080/db/v1/tables/hello \
  -H "Content-Type: application/json" \
  -d '{}'

curl -fsS -X POST http://127.0.0.1:8080/db/v1/tables/hello/batch \
  -H "Content-Type: application/json" \
  -d '{"inserts":{"first":{"title":"Hello","body":"Antfly is ready."}},"sync_level":"full_text"}'

antfly query --table hello \
  --full-text-search 'body:Antfly' \
  --fields 'title,body'

The last command returns the document with key first. If it does not, stop here and see Troubleshooting.

Install the text embedding model#

Pull the text embedding model used for semantic search. Antfly Inference runs it inside the same process, with no external API:

antfly inference pull hf:Qwen/Qwen3-Embedding-0.6B-GGUF:q8-0-bundle-v1
antfly inference list

Qwen3-Embedding uses an instruction for search queries but embeds indexed documents without it. Antfly applies that distinction automatically: index and artifact writes use the document role, while semantic searches use the query role. The Q8_0 bundle runs on native CPU on Linux and macOS, and uses Metal automatically when it is available.

If the running process does not pick up the new model, restart it once before creating the index.

Once the embeddings index reaches queryable, Antfly durably preserves that published snapshot across later restarts while the remaining embeddings resume in the background. complete still means all source outcomes and the final vector publication have settled.

Download Sample Data#

In a new terminal, download 10,000 Wikipedia articles enriched with thumbnail images:

curl -L -o wiki-articles.jsonl https://cdn.antfly.io/datasets/wiki-articles-10k-v001.json
head -n 1 wiki-articles.jsonl | jq '.'

One article per line, each with title, body, and url. 2,446 of the 10,000 also carry a thumbnail_url, which the image search step uses later.

Create a Table#

Create a table with a text embedding index for semantic search:

antfly table create --table wikipedia \
  --index '{
    "name": "title_body",
    "type": "embeddings",
    "template": "{{title}} {{body}}",
    "embedder": {
      "provider": "antfly",
      "model": "Qwen/Qwen3-Embedding-0.6B-GGUF:q8-0-bundle-v1"
    },
    "chunker": {
      "provider": "antfly",
      "text": {
        "target_tokens": 200,
        "overlap_tokens": 25
      }
    }
  }'

Verify the table and index were created:

antfly table list
antfly index list --table wikipedia

title_body is the embeddings index you created. full_text_index_v0 came free: every table gets one, and it is what the keyword query below runs against.

Load Data#

# Load all 10,000 articles using the title as the document ID
antfly load --table wikipedia \
  --file wiki-articles.jsonl \
  --id-field title \
  --sync-level full_text

The CLI reads the JSONL and batches it for you. --sync-level full_text returns once each batch is keyword-searchable; embeddings keep generating in the background. (An application that owns its own ingestion uses the SDK batch operations with bounded batches and retries.)

Embedding generation runs in the background. Wait until 10% of documents have searchable embeddings; skipped documents are excluded:

antfly index wait --table wikipedia \
  --index title_body \
  --until source-covered=10% \
  --timeout 45m

Continue as soon as it returns. Results improve as the remaining documents are indexed; antfly index list --table wikipedia shows progress. To wait for the full corpus instead, use --until complete. Timing depends on hardware. If the wait times out, do not press on; see Troubleshooting.

Articles can produce multiple chunks, so SEARCHABLE in the status output may exceed the document count, and avg_embeddings measures chunk embeddings per second, not documents.

Full-Text Search#

Search for articles containing specific terms:

antfly query --table wikipedia \
  --full-text-search 'body:"Korea"' \
  --fields "title,url" \
  --limit 5

Find articles semantically similar to a natural language query:

antfly query --table wikipedia \
  --semantic-search "anatomy and physiology" \
  --indexes "title_body" \
  --fields "title,url" \
  --limit 5

Hybrid Search & Reranking#

One query can run the keyword and semantic searches together; Antfly fuses the two ranked lists with Reciprocal Rank Fusion. The Qwen3 text reranker then rescores each query-document pair to improve the ordering of those candidates.

Pull the reranker model:

antfly inference pull hf:ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF:gguf:Q8_0 --tasks rerank

The Q8 model is about 610 MiB and runs on native CPU or Metal. The first query loads it; reranking full articles can take tens of seconds, especially while indexing continues.

antfly query --table wikipedia \
  --full-text-search 'body:Einstein' \
  --semantic-search "theory of relativity and physics" \
  --indexes "title_body" \
  --fields "title,url" \
  --limit 10 \
  --reranker '{
      "provider": "antfly",
      "model": "ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF:gguf:Q8_0",
      "field": "body"
    }' \
  --pruner '{"min_score_ratio": 0.01}'

Reranking changes the order and keeps the count. Pruning changes the count: min_score_ratio keeps only results scoring at least that fraction of the top result, and max_score_gap_percent stops at the first sharp drop. Tune Hybrid Search covers both in depth.

ClipClap places text and images in one embedding space, so you can describe a picture in words and get the articles whose thumbnails match.

Confirm the ClipClap model is available#

antfly inference pull antflydb/clipclap:gguf:Q4_K

Add an image index to your table#

Add a ClipClap-based index to embed each article's Wikipedia thumbnail:

antfly index create --table wikipedia \
  --index thumbnail \
  --type embeddings \
  --coverage-policy partial \
  --template "{{#if thumbnail_url}}{{remoteMedia url=thumbnail_url}}{{/if}}" \
  --embedder '{"provider": "antfly", "model": "antflydb/clipclap"}' \
  --distance-metric cosine

Antfly probes the selected embedding model during creation and stores its dimension in the normalized index definition. Specify dimension only when you also want creation to verify an explicit expected size.

Adding an index starts background image indexing. Articles without thumbnails are skipped; wait for 250 searchable images before continuing.

antfly index wait --table wikipedia \
  --index thumbnail \
  --until searchable-artifacts=250 \
  --timeout 45m

Search by visual content#

Describe what you want to see:

antfly query --table wikipedia \
  --semantic-search "map of a country" \
  --indexes "thumbnail" \
  --fields "title,url,thumbnail_url" \
  --limit 5

Concrete visual descriptions ("red sports car", "mountain landscape") work better than abstract ones. To search with an image instead of words, see Multimodal Embeddings.

RAG (Retrieval-Augmented Generation)#

The retrieval agent plans the query, runs the search, and hands the results to a local language model to write the answer.

Pull a generation model#

antfly inference pull hf:ggml-org/gemma-4-E4B-it-GGUF:gguf:Q4_0 \
  --tasks generate \
  --projector auto

--projector auto includes image and audio support; try a local file with antfly inference generate <model-dir> "Describe this input" --image ./photo.jpg (antfly inference list prints the model directory) or --audio ./clip.wav, or use --projector none for a smaller, text-only download. The RAG example below uses text and limits output to 512 tokens per model turn. Hosted providers are one section down.

Build a query and generate an answer#

Give Gemma an intent and a table. The retrieval agent delegates query planning to the query builder, searches, and answers from the results.

antfly agents retrieval --table wikipedia \
  --intent "What are the major events in Korean history?" \
  --fields "title,body" \
  --limit 5 \
  --generator '{
    "provider": "antfly",
    "model": "ggml-org/gemma-4-E4B-it-GGUF",
    "max_tokens": 512
  }' \
  --max-internal-iterations 8 \
  --streaming

The CLI and cURL commands stream Server-Sent Events as each step completes. A terminal event: done marks success; an error or incomplete stream makes the CLI exit nonzero. The SDK examples request structured JSON instead. Each example uses bounded, read-only tool calls and requests only a grounded answer. Use --no-streaming for JSON from the CLI.

Inspect a query plan (optional)#

To build a query without retrieving documents:

antfly agents query-builder --table wikipedia \
  --intent "Find articles about anatomy" \
  --generator '{"provider":"antfly","model":"ggml-org/gemma-4-E4B-it-GGUF","max_tokens":512,"temperature":0}' \
  --max-internal-iterations 8

Add --execute to run the same composed retrieval workflow shown above.

RAG with alternative providers#

The same request runs against a hosted model or Ollama by changing the generator:

Requires a Google AI API key. The key is read by the standalone process, not the CLI, so export it in the terminal that runs antfly standalone and restart it:

export GEMINI_API_KEY="your-api-key-here"

antfly agents retrieval --table wikipedia \
  --semantic-search "What are the major events in Korean history?" \
  --indexes "title_body" \
  --fields "title,body" \
  --limit 5 \
  --generator '{"provider": "gemini", "model": "gemini-2.5-flash"}' \
  --max-context-tokens 2048 \
  --generate

Any of the four does the job. The two local options need no key and keep the data on the machine; the two hosted ones are faster on a laptop without a GPU.

Structured output#

--no-streaming returns one JSON body with the answer and its sources instead of streamed text:

antfly agents retrieval --table wikipedia \
  --semantic-search "Explain the theory of relativity" \
  --indexes "title_body" \
  --fields "title,body,url" \
  --limit 5 \
  --generator '{"provider": "gemini", "model": "gemini-2.5-flash"}' \
  --no-streaming --generate

Evaluation#

steps.eval scores retrieval and generation quality in the same request. The CLI has no evaluation flag, so send it on the API request:

curl -N -X POST http://127.0.0.1:8080/db/v1/agents/retrieval \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -d '{
    "query": "What are the major events in Korean history?",
    "queries": [{
      "table": "wikipedia",
      "semantic_search": "What are the major events in Korean history?",
      "indexes": ["title_body"],
      "fields": ["title", "body"],
      "limit": 5
    }],
    "generator": {"provider": "gemini", "model": "gemini-2.5-flash"},
    "stream": true,
    "steps": {
      "generation": {"enabled": true},
      "eval": {
        "evaluators": ["faithfulness", "relevance"],
        "judge": {"provider": "gemini", "model": "gemini-2.5-flash"}
      }
    }
  }'

The scores arrive as an eval event in the stream.

The evaluators:

  • Retrieval metrics (require ground_truth.relevant_ids): recall, precision, ndcg, mrr, map
  • LLM-as-judge metrics (require judge config): relevance, faithfulness, completeness, coherence, safety, helpfulness, correctness, citation_quality

Troubleshooting#

antfly is not found after the Linux install#

The installer puts the binary in ~/.local/bin. Add it to the current shell and retry:

export PATH="$HOME/.local/bin:$PATH"
antfly --version

Antfly reports a legacy data directory#

Antfly will not overwrite a data directory from the old Go runtime. Leave it in place, back it up with that runtime if you need it, and start this quickstart on a fresh directory:

antfly standalone --data-dir "$HOME/.antfly-zig" 2>&1 | tee antfly.log

Models stay in ~/.antfly/inference/models unless you pass --models-dir or set ANTFLY_INFERENCE_MODELS_DIR.

MODEL_NOT_FOUND#

Confirm the model is installed and check the model directory the startup log printed:

antfly inference list
grep -F 'standalone inference paths' antfly.log

Pull it again, or point at the right directory with antfly standalone --models-dir <path>.

An index remains pending or index wait times out#

Wait for the readiness threshold each step names before moving on. queryable_partial is safe to query while coverage grows, but an empty published generation cannot produce results. If index wait times out, capture the full status and the recent log:

antfly index list --table wikipedia --output json | jq '.'
grep -E 'watchdog|derived worker|NotFound|EnrichmentWaitTimeout' antfly.log | tail -n 100

Chunk and vector counts exceeding the article count is normal. Counts that keep growing on unchanged data, coverage that never advances, or a ready index dropping back to pending is a bug: report it with the JSON status, the log excerpt, the Antfly version, and the startup command.

Startup reports a port conflict or memory error#

Stop whatever holds ports 8080 or 4200, or pick others with --port and --health-port. For memory errors, stop other large local models, or retry with a smaller model or a larger --inference-host-budget-mb.

Stop the quickstart#

Press Ctrl+C in the standalone terminal, or with Homebrew services:

brew services stop antfly

Give Your Agent the Antfly Skill#

Everything you just did by hand, a coding agent can do for you. The Antfly skill carries verified, current knowledge of Antfly's APIs, query shapes, React components, and operations, for Claude Code, Cursor, Codex, and other agent harnesses:

npx skills add antflydb/antfly-skills

From here your agent can create tables, build indexes, and write queries against the instance you just started.

Next Steps#

You have a running instance, an indexed table, and an agent that knows Antfly. Each of these guides builds one thing on top of that and ends with a prompt you can hand to the agent:

For production, the Antfly Operator deploys on Kubernetes, and backup and restore protects what you loaded.