Gemma 4 QAT
Generate answers, summaries, and text with the instruction-tuned Gemma 4 E4B checkpoint. Quantization-aware training prepares the model for Q4_0 weights, reducing the memory needed for local generation.
antfly inference pull google/gemma-4-E4B-it-qat-q4_0-gguf:ggufTechnical Specs
- Architecture
- Gemma 4 E4B
About
Gemma 4 E4B is Google's instruction-tuned model with 4.5B effective parameters (8B including embeddings). It takes text, images, and audio as input and has a 128K-token context window. This QAT checkpoint was trained for 4-bit Q4_0 weights, and the pull also fetches its multimodal projector, so image and audio inputs work without a separate download.
Choose E4B over Gemma 4 E2B QAT (2.3B effective parameters) when answer quality matters more than memory and speed. E2B is the pick for smaller machines and faster replies.
Capabilities
- Answers, summaries, and chat from text, image, and audio inputs
- 128K-token context window
- Out-of-the-box support for more than 35 languages
- 4-bit Q4_0 weights from quantization-aware training
Where it fits in Antfly
Gemma 4 is the generator for answers and agents. Set it as the generator of a retrieval agent and it writes the answer from the hits Antfly retrieves. Because it accepts images, it can also stand in as the OCR reader for scanned pages in a document_extraction enrichment.
See Build a Support Answer Agent for a generator in a retrieval agent, and Antfly Inference for pulling and configuring models.
Usage
Pull and start Antfly
antfly inference pull google/gemma-4-E4B-it-qat-q4_0-gguf:gguf
antfly standaloneGenerate text with Gemma 4
Send a conversation to the generation endpoint. This example limits the response to 128 tokens.
curl http://localhost:8080/ai/v1/generate \
-H 'Content-Type: application/json' \
-d '{
"model": "google/gemma-4-E4B-it-qat-q4_0-gguf:gguf",
"messages": [
{
"role": "user",
"content": "Explain hybrid search in two sentences."
}
],
"max_tokens": 128
}'