← Back to models

Qwen3 Reranker

reranker

Reorder search results by relevance to the query, scoring each query and document pair with a compact multilingual model.

antfly inference pull ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF:gguf:Q8_0
By QwenLicense: apache-2.0Hugging Face source

Technical Specs

Architecture
Qwen3
Parameters
0.6B
Q8_0GGUFmultilingual

Usage

Pull the model

antfly inference pull ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF:gguf:Q8_0

Start Antfly Inference

# With Antfly standalone (Antfly Inference enabled by default)
antfly standalone

# Or run Antfly Inference on its own, listening on 127.0.0.1:8090
antfly inference run

Use in configuration

{
  "rerankers": {
    "local": {
      "provider": "antfly",
      "model": "ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF:gguf:Q8_0"
    }
  }
}