Antfly
The built-in inference runtime, which runs every component locally beside the database
The antfly provider is Antfly's own inference runtime. In antfly standalone
it runs inside the database process; it can also run as a separate
antfly inference run service or a Kubernetes inference pool and be reached
over HTTP. Data never leaves your machines, and there is no per-call cost.
Antfly Inference covers how the runtime is deployed and
configured.
What It Provides
| Component | Inputs | Recommended model |
|---|---|---|
| Embedder | Text, images, audio (model-dependent) | BAAI/bge-small-en-v1.5 for text, antflydb/clipclap for text, image, and audio |
| Reranker | Text | ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF |
| Chunker | Text | fixed_bert (built in) |
| Reader | Images and documents | lightonai/LightOnOCR-3-0.8B |
| Transcriber | Audio | openai/whisper-small; openai/whisper-tiny for clean dictation |
| Generator | Text, images, audio, video (model-dependent) | ggml-org/gemma-4-E4B-it-GGUF |
| Extractor | Text | antflydb/gliner2-base-v1 |
| Decider | Text | Laya typed decisions, or EmbeddingGemma 2 similarity |
Images and audio reach a generator when the model ships a vision or audio projector; Gemma 4 E4B accepts both, and Gemma 4 also takes video prompts. Local Model Compatibility lists every verified model and alternate, and Model Downloads explains formats and pinned revisions.
Credentials
None when the runtime is embedded. A remote Antfly inference service accepts an
api_key, which defaults to ANTFLY_INFERENCE_API_KEY.
Example
antfly inference pull hf:BAAI/bge-small-en-v1.5 --tasks embed
{
"embedders": {
"local": { "provider": "antfly", "model": "BAAI/bge-small-en-v1.5" }
}
}
Set api_url to reach a separate inference service instead of the embedded
runtime.
Related
- Antfly Inference: deployment, model storage, and capacity
- LightOnOCR-3 and Index Recorded Calls and Podcasts: reading and transcription
- Inference runtime reference: admission, request ceilings, and model instances