Model Downloads

antfly inference pull requires a Hugging Face owner/model reference. The optional hf: prefix is accepted. Revision and format are independent:

antfly inference pull BAAI/bge-m3
antfly inference pull BAAI/bge-m3@main
antfly inference pull BAAI/bge-m3:onnx@main
antfly inference pull owner/model:onnx@release/v2
antfly inference pull 'BAAI/bge-m3:safetensors@84790c1a606f60d06c6932e4ecdd174b466d84ac'
antfly inference pull gpustack/bge-m3-GGUF:gguf:Q8_0

Omitting the revision means main. Branches, tags, and commit hashes are accepted. At the start of each explicit pull, the Hub's metadata resolves that request to one immutable commit. Metadata, weights, tokenizer, and configuration all come from that snapshot, even if the branch changes while the transfer is running. For managed AI models, the completion receipt records the requested revision, resolved commit, and selected format. Repeating a pull resolves moving refs again and reuses only verified files; inference uses the installed snapshot without network refresh.

Traditional ML predictor pulls also resolve one snapshot before downloading, including explicit branches or commits:

antfly inference pull hf:owner/tree-model@release/v2 --type predictor

Their selected artifact's size and available SHA-256 or Git blob digest are verified before conversion and atomic installation.

Pull rejects all short aliases, including names with format or revision suffixes. Qwen shortcuts such as qwen3-embedding and qwen3-vl-2b are no longer expanded by inference or chat. Legacy BGE and Gemma shortcuts remain available for runtime lookup and chat. Explicit canonical Qwen bundle references, such as Qwen/Qwen3-Embedding-0.6B-GGUF:q8-0-bundle-v1, still select their immutable multi-repository bundles.

Format selection#

An explicit :gguf, :gguf:Q8_0, :safetensors, or :onnx request either selects that format or reports the available formats. An unknown variant is an error. :f32 and :i8 retain their ONNX meanings. There is no implicit fallback from an explicit ONNX request to safetensors.

The legacy :native selector is an alias for :auto, including revision suffixes such as :native@main. It selects artifacts and does not force an execution backend. Receipts retain the requested selector and record the selected format.

Automatic selection chooses one artifact set: GGUF first, then safetensors, then ONNX. :hybrid explicitly requests both ONNX and safetensors when available. Selection depends on repository artifacts, not the machine that downloads them. Architecture/operator support and the CPU, Metal, CUDA, or ONNX Runtime execution backend are checked when loading the model; a downloadable file is not a claim that every backend can execute it.

Safetensors shards are selected from the index's weight_map, including arbitrary shard names. ONNX external data is discovered from initializers, tensor attributes, nested graphs, sparse tensors, and functions. Missing references and paths that escape the model root fail the pull before publication. Directory structure is preserved. For managed ONNX exports, export-local configuration and tokenizer files take precedence over repository-root equivalents.

Size and digest checks, bounded downloads, resumable staging, and atomic publication remain in force. A failed pull leaves the previously installed model available. Do not work around a mismatch by disabling verification or changing the expected size.

PyTorch .bin/.pt, TensorFlow, and Flax weights are not executable formats for this pull path. Use an explicitly converted, supported artifact set; pull does not execute repository code or automatically convert framework checkpoints.

BGE-M3#

The official repository's current main contains ONNX and PyTorch weights, but no GGUF or safetensors. Automatic selection therefore uses its ONNX export. The historical safetensors reference above remains available explicitly; community GGUF repositories must also be selected explicitly.

The embedding qualification covers dense, normalized CLS embeddings. The presence of sparse_linear.pt or colbert_linear.pt in the upstream repository does not advertise sparse or multi-vector support for the selected dense export.

The M4 Max qualification report records accuracy, batching, CPU/Metal parity, and latency for the three formats. Imported ONNX uses cached plans for concrete input shapes, shared resident weights, reusable GPU buffers, and bounded Metal command frames. Qualification checks actual batch execution and rejects CPU fallback, intermediate host outputs, and device transfers. On the measured short-input workload, all three formats benefit from Metal; the report states the tested batch sizes and context lengths.

Run the small downloader suite from zig/ with:

zig build inference-registry-test -Doptimize=ReleaseFast

The CPU/Metal format qualification uses an independent ONNX Runtime oracle and compares token IDs, normalized embeddings, mixed-length batching, and throughput:

# From zig/:
zig build inference-build-bge-m3-benchmark -Doptimize=ReleaseFast

# From the repository root; directories must contain completed model pulls:
uv run --script zig/pkg/inference/scripts/embedder/bge-m3/qualify_formats.py \
  --binary zig/zig-out/bin/antfly-inference-bge-m3-e2e-bench \
  --oracle-dir /path/to/official-bge-m3 \
  --model onnx=/path/to/official-bge-m3 \
  --model safetensors=/path/to/historical-bge-m3 \
  --model gguf=/path/to/bge-m3-gguf \
  --batches 1,2,4,8,16 --require-resident-onnx \
  --output /tmp/bge-m3-qualification.json

Use --texts-json PATH to qualify an additional JSON array of texts; the checked-in unicode-corpus.json includes combining marks, emoji sequences, and non-Latin scripts. --graph-runtime partitioned checks the explicit partitioned strategy; interpreter is also available, without the residency requirement.

For an independent execution reference for the community BGE-M3 Q8 GGUF, add --oracle-gguf /path/to/bge-m3-Q8_0.gguf --minimum-cosine 0.9999. This replaces every weight in the official ONNX graph with gguf-py's dequantized values and executes it through ONNX Runtime. The default oracle still uses the official full-precision weights, so conversion accuracy and runtime parity are separate checks. Neither mode silently relaxes a failed threshold.