← Back to models

Gemma 4 QAT

generatorAnswers

Generate answers, summaries, and text with the instruction-tuned Gemma 4 E4B checkpoint. Quantization-aware training prepares the model for Q4_0 weights, reducing the memory needed for local generation.

antfly inference pull google/gemma-4-E4B-it-qat-q4_0-gguf:gguf
By googleLicense: apache-2.0Hugging Face source

Technical Specs

Architecture
Gemma 4 E4B
QATQ4_0GGUF

About#

Gemma 4 E4B is Google's instruction-tuned model with 4.5B effective parameters (8B including embeddings). It takes text, images, and audio as input and has a 128K-token context window. This QAT checkpoint was trained for 4-bit Q4_0 weights, and the pull also fetches its multimodal projector, so image and audio inputs work without a separate download.

Choose E4B over Gemma 4 E2B QAT (2.3B effective parameters) when answer quality matters more than memory and speed. E2B is the pick for smaller machines and faster replies.

Capabilities#

  • Answers, summaries, and chat from text, image, and audio inputs
  • 128K-token context window
  • Out-of-the-box support for more than 35 languages
  • 4-bit Q4_0 weights from quantization-aware training

Where it fits in Antfly#

Gemma 4 is the generator for answers and agents. Set it as the generator of a retrieval agent and it writes the answer from the hits Antfly retrieves. Because it accepts images, it can also stand in as the OCR reader for scanned pages in a document_extraction enrichment.

See Build a Support Answer Agent for a generator in a retrieval agent, and Antfly Inference for pulling and configuring models.

Usage#

Pull and start Antfly#

antfly inference pull google/gemma-4-E4B-it-qat-q4_0-gguf:gguf

antfly standalone

Generate text with Gemma 4#

Send a conversation to the generation endpoint. This example limits the response to 128 tokens.

curl http://localhost:8080/ai/v1/generate \
-H 'Content-Type: application/json' \
-d '{
"model": "google/gemma-4-E4B-it-qat-q4_0-gguf:gguf",
"messages": [
  {
    "role": "user",
    "content": "Explain hybrid search in two sentences."
  }
],
"max_tokens": 128
}'