← Back to models

Gemma 3

generatorAnswers

Generate answers and text with the instruction-tuned Gemma 3 4B model.

antfly inference pull unsloth/gemma-3-4b-it-GGUF:gguf:Q4_K_M
By googleLicense: gemmaHugging Face source

Technical Specs

Architecture
Gemma 3
Parameters
4B
Q4_K_MGGUF

Usage

Pull the model

antfly inference pull unsloth/gemma-3-4b-it-GGUF:gguf:Q4_K_M

Start Antfly Inference

# With Antfly standalone (Antfly Inference enabled by default)
antfly standalone

# Or run Antfly Inference on its own, listening on 127.0.0.1:8090
antfly inference run

Use in configuration

{
  "generators": {
    "local": {
      "provider": "antfly",
      "model": "unsloth/gemma-3-4b-it-GGUF:gguf:Q4_K_M"
    }
  }
}