← Back to models

Llama 3.2

generator

Complete and continue text with Meta's Llama 3.2 1B model.

antfly inference pull unsloth/Llama-3.2-1B-Instruct-GGUF:gguf:Q4_K_M
By meta-llamaLicense: llama3.2Hugging Face source

Technical Specs

Architecture
Llama 3.2
Parameters
1.2B
Q4_K_MGGUF

Usage

Pull the model

antfly inference pull unsloth/Llama-3.2-1B-Instruct-GGUF:gguf:Q4_K_M

Start Antfly Inference

# With Antfly standalone (Antfly Inference enabled by default)
antfly standalone

# Or run Antfly Inference on its own, listening on 127.0.0.1:8090
antfly inference run

Use in configuration

{
  "generators": {
    "local": {
      "provider": "antfly",
      "model": "unsloth/Llama-3.2-1B-Instruct-GGUF:gguf:Q4_K_M"
    }
  }
}