Antfly Lite

How the embedded single-file profile of Antfly works, what it shares with a normal instance, and how data moves between the two

Intermediate20 min

liteembeddedlocal-firstbackupmigration

Prerequisites

Antfly Lite CLI built or installed

Antfly Lite is Antfly in one file. Documents, schemas, indexes, enrichments, and query-visible artifacts live in a local .aflite database that an application opens directly, with no server process. Choose Lite for development, desktop apps, edge workers, tests, and single-user deployments. Choose a normal instance when you need replication, shard placement, multi-node scaling, or cluster metadata. Everything else, from the index model to the query API, is the same code.

Embedded storage query execution rejects non-null evaluate; use the supported embedded API surface for local queries. The file-format revision below is independent of the Antfly release version.

antfly lite init app.aflite
antfly lite status app.aflite

init refuses to overwrite an existing path. status reports the storage identity and the inference state:

{
  "storage": {
    "format": "aflite",
    "engine": "native_single_file",
    "format_version": 4
  },
  "inference": {
    "mode": "caller_supplied_or_disabled",
    "available_modes": [
      "caller_supplied_artifacts",
      "remote_provider",
      "local_embedded",
      "disabled_deferred"
    ],
    "configured": false,
    "no_inference_configured_ok": true
  }
}

Two File Types, Two Jobs#

Extension	What it is	Use it for
`.aflite`	The live embedded database	The file your application opens
`.afb`	An Antfly Backup Bundle	Archival retention, object-storage uploads, repeatable migrations, restoring into a normal instance

An .afb bundle's manifest names its representation: portable (logical records) or native (physical generations). Lite writes and reads the portable representation; a normal instance can also package native generations in the same envelope. AFB1 portable archives remain readable, and new bundles use AFB2, which declares the representation explicitly. The Lite backup and export commands emit a self-contained full bundle. AFB2 also defines delta bundles against an exact base for incremental repository workflows; a delta is never treated as a self-contained archive.

Never treat .aflite as the backup format. A normal Antfly can restore directly from one, but that is a convenience for promotion, not an archive. The .aflite open path accepts the documented native single-file format and fails explicitly on unknown versions or invalid headers; pre-release directory and LSM-container experiments are not read. To keep one of those, export it with the matching development build and restore the resulting portable AFB1 or AFB2 bundle.

Writes, Reads, and Indexes#

Requests are the same JSON shapes as the normal API, passed as files. A batch write:

{
  "inserts": {
    "doc:lite:1": {
      "title": "Local Antfly",
      "body": "Antfly Lite stores this document in app.aflite."
    }
  },
  "sync_level": "full_index"
}

antfly lite batch app.aflite --file writes.json
antfly lite lookup app.aflite --key doc:lite:1

A key-range scan (scan.json):

{
  "from": "doc:",
  "to": "doc;",
  "include_documents": true,
  "limit": 10
}

antfly lite scan app.aflite --file scan.json

Indexes use the same logical model as a normal instance. A full-text index (full-text-index.json):

{
  "name": "body_text",
  "kind": "full_text",
  "config_json": "{\"fields\":[\"title\",\"body\"]}"
}

antfly lite index create app.aflite --file full-text-index.json
antfly lite run-until-idle app.aflite

A query against it (query.json). Note that the antfly lite CLI's query JSON takes a nested match: {field, text} object here, unlike the public /db/v1 API's flat match: {match: string, field: string} shape used elsewhere in the docs:

{
  "full_text_search": {
    "match": {
      "field": "body",
      "text": "local antfly"
    }
  },
  "limit": 5
}

antfly lite query app.aflite --file query.json

Dense vector, sparse vector, graph, schema, and enrichment catalog operations are all available through the same antfly lite CLI and the embedded APIs.

Inference Is Optional#

A Lite database with no inference configured is a complete, healthy state. It can store caller-supplied embeddings and sparse vectors, index and query them, hold enrichment definitions, and report through status whether inference-backed work is possible. A fresh database reports configured: false with no_inference_configured_ok: true.

The mode in the status inference block tells an application which path it is on:

Mode	Meaning
`caller_supplied_artifacts`	The application writes embeddings, sparse vectors, assets, chunks, and graph edges itself
`remote_provider`	Lite calls an inference provider the application configured at open time
`local_embedded`	A full build ships an embedded inference runtime
`manual_maintenance`	A hosted profile exposes pending work for the application to drain with `run-until-idle`
`disabled_deferred`	Definitions and documents are stored; model-backed work waits until a runtime is available

Applications that configure a remote provider or embed a local runtime mark it at open time. Status then reports configured: true with remote_provider_configured: true or local_runtime_configured: true and local_runtime_available: true, so bindings branch on status rather than on enrichment failures. The capabilities block also carries retrieval flags (text_search, dense_vector_search, sparse_vector_search, hybrid_search, graph_search, caller_supplied_embeddings) for the same purpose.

antfly lite run-until-idle app.aflite drains whatever local maintenance work is available. Run it after writes and before a query or a backup when you want that work reflected.

The Same API Over HTTP#

For SDKs and local tools, the full standalone database server can serve a Lite file through its HTTP API:

antfly standalone --storage-engine lite --storage-path app.aflite
curl http://127.0.0.1:8080/db/v1/status

HTTP serving uses antfly standalone; antfly lite manages files, and libantfly provides the in-process API. All three use the same Apache-2.0 engine.

Direct adoption currently supports the embedded root as the default table, with its schema, full-text index configurations and supported enrichments. Native dense, sparse, graph and algebraic indexes, resolver catalogs, and additional embedded tables are rejected before adoption. Stored embeddings alone do not imply a supported ANN index.

Adoption requires the public JSON contract for an existing schema and verifies that it matches the stored runtime schema. A binary-only schema written through native DB.setSchema() is rejected before ownership transfers. While the file is still embedded-owned, publish the intended public schema at a newer version through setSchemaJson() or lite schema set, then retry adoption. The runtime schema cannot be inverted into a public contract without losing semantics.

Once adopted, the standalone catalog owns schema, index, enrichment, resolver and table definitions. Embedded clients can still read, search and edit documents offline, but definition changes must go through the standalone API. This rule also applies to SQL DDL and persists in physical snapshots, keeping one catalog authoritative when a returned file is served again.

Ordinary schema migrations preserve single-file handoff: stop the writer, copy the file, and reopen it with a fresh standalone data directory. Schema cleanup reuses and validates FK publication authority only when publication admission already acquired it. Files with that directory-bound FK authority still require the associated physical root; copying the file alone does not transfer hidden owner authority.

Backups, Snapshots, and Imports#

antfly lite backup app.aflite --out app.afb        # export is an alias
antfly lite restore app.afb --out restored.aflite
antfly lite snapshot app.aflite --out copy.aflite

backup writes a self-contained AFB2 bundle with the portable representation. Its manifest maps logical records to unique content digests, and a fixed trailer locates the digest index so a restore runs in bounded memory. A delta bundle, when a repository or export workflow supplies one, names the exact base-manifest digest it depends on and fails closed if that base is unavailable. restore creates a new Lite database from a bundle. snapshot copies a stable live checkpoint of the .aflite file itself, which is the right tool for local copies, test fixtures, and handing a database to another Lite application.

import loads a portable AFB1 or AFB2 bundle into a database that already exists:

antfly lite init empty.aflite
antfly lite import empty.aflite --from app.afb
antfly lite import restored.aflite --from app.afb --replace

--replace is required when the target holds data. Importing from another .aflite is a physical snapshot replacement, never a logical merge.

Moving Between Lite and a Normal Instance#

Directory tables using dense_embeddings: vector_store support native and portable cohort backups once storage migration has finished. Native backups retain the exact source-vector generation with the primary checkpoint; portable backups resolve those references into inline vectors. Restoring a portable backup therefore provides a path back to primary_lsm storage without recomputing embeddings. Backups are rejected while a migration is active so a restore cannot mix physical generations.

Both directions are logical backup and restore. The portable stream carries documents and keys, schemas, index definitions, enrichment definitions and persisted state, caller-supplied vectors and artifacts that are already query-visible, and enough metadata to rebuild query-visible indexes. Storage-engine files (text indexes, dense vector segments, sparse postings, graph reverse indexes, checkpoints, free-space metadata) are rebuilt on the target, not copied.

Lite into a running instance. promote opens the .aflite read-only, stages a portable backup, restores it into the target table, and leaves the Lite file in place:

antfly lite promote app.aflite \
  --target http://127.0.0.1:8080 \
  --table docs \
  --connection archive-writer \
  --location file:///tmp/antfly_backups

antfly restore --input app.aflite --table docs --connection archive-writer --location file:///tmp/antfly_backups --url http://127.0.0.1:8080 does the same thing through the normal restore command. Either way --connection and --location are required: the target server reads the staged backup through that named connection, from a shared file://, s3://, or gs:// location it can reach. For a repeatable release or disaster-recovery workflow, make the .afb explicit first:

antfly lite backup app.aflite --out app.afb
antfly restore \
  --input app.afb \
  --table docs \
  --connection archive-writer \
  --location file:///tmp/antfly_backups \
  --url http://127.0.0.1:8080

The normal restore --input command also accepts .afb2. Preparation detects Lite and AFB formats from their contents, validates portable records with the restore reader, and streams files through temporary disk storage. Portable backups retain their bytes; Lite exports retain the metadata from the same snapshot as their documents. A native AFB2 bundle keeps its native representation. Delta bundles need their base and are not supported by this input path.

For an offline source inventory and compatibility report, use:

antfly backup inspect --input app.aflite
antfly backup inspect --input app.afb --destination-capabilities destination.json

Inspection runs without a server connection. Exit success means a JSON report was produced; check completeness and compatibility.verdict before using it. It does not reserve a destination or validate a live restore request. See Backup inspection for the report and capability-attestation contract.

The default preparation limits are 64 GiB of input and 128 GiB per prepared artifact or expanded logical stream. These are safety limits, not a guarantee that a destination has enough capacity. Allow disk space for the input, temporary copies, and native extraction. Format validation does not check whether the destination can access media or run the configured models; the shared location and named connection are still required. Preparation places scratch files and native extraction directories under TMPDIR, then TEMP, falling back to /tmp when neither is set.

A normal instance into Lite. Back up the table with --format portable (Lite cannot read a native bundle), then restore or import it:

antfly backup \
  --table docs \
  --backup-id docs-export \
  --connection archive-writer \
  --format portable \
  --out docs.afb \
  --location file:///tmp/antfly_backups \
  --url http://127.0.0.1:8080

antfly lite restore docs.afb --out docs.aflite

Lite is a single-node, single-shard source, so restoring into a normal instance maps it into the target table's placement model, and restoring the other way collapses the selected table into one file. This is how you build fixture databases, debug a production slice locally, or ship a seeded embedded application.

After either direction, drain and verify on the Lite side:

antfly lite status docs.aflite
antfly lite run-until-idle docs.aflite
antfly lite check docs.aflite

On a normal target, use its status and restore verification. The migration is complete when pending index and enrichment maintenance is drained or explicitly deferred. Caller-supplied artifacts arrive as stored data and stay queryable; deferred model-backed work stays pending until a runtime is available on the destination. Run run-until-idle before taking a migration backup when you want all locally available maintenance reflected in it.

Embedding Lite in Zig and Go#

Zig applications use the antfly-embedded package. db.DB.createLite makes a native .aflite, db.DB.openLite opens one, and package-level helpers inspect and snapshot files without a DB handle:

const antfly = @import("antfly-embedded");

var db = try antfly.db.DB.createLite(allocator, "zig-app.aflite", .{});
defer db.close();

const report = try antfly.db.checkLiteFile(allocator, "zig-app.aflite");
_ = report.valid;

_ = try antfly.db.copyStableLiteSnapshotFile(
    allocator,
    "zig-app.aflite",
    "zig-copy.aflite",
    false,
);

antfly.api.checkLiteFileJson and antfly.api.copyStableLiteSnapshotFileJson return the same JSON reports the CLI and C ABI expose.

Go, Python, Rust, and TypeScript bindings wrap the same libantfly C ABI. In Go:

go get github.com/antflydb/antfly/go/pkg/embedded

package main

import (
	"log"
	"os"

	"github.com/antflydb/antfly/go/pkg/embedded"
)

func main() {
	// Create on first run; Open never creates or upgrades a file.
	open := embedded.Open
	if _, err := os.Stat("go-app.aflite"); os.IsNotExist(err) {
		open = embedded.Create
	}
	db, err := open("go-app.aflite")
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	err = db.Batch([]embedded.WriteIntent{{
		Key:   "doc:go:1",
		Value: []byte(`{"title":"Embedded Lite","body":"Antfly Lite keeps data in one local file."}`),
	}}, 1)
	if err != nil {
		log.Fatal(err)
	}
	if _, err := db.RunUntilIdleStatus(); err != nil {
		log.Fatal(err)
	}

	doc, err := db.LookupJSON("doc:go:1")
	if err != nil {
		log.Fatal(err)
	}
	log.Printf("lookup: %s", doc)

	hits, err := db.SearchJSON([]byte(`{"full_text_search":{"match":{"field":"body","text":"local file"}},"limit":3}`))
	if err != nil {
		log.Fatal(err)
	}
	log.Printf("search: %s", hits)

	if err := db.BackupToFile("go-app.afb"); err != nil {
		log.Fatal(err)
	}
}

Create is the explicit first-use path; Open takes an existing file and never creates or upgrades one. Constants such as embedded.InferenceModeCallerSuppliedArtifacts and embedded.InferenceModeDisabledDeferred let you branch on status without hard-coding strings. Building from the source tree means building the C ABI first (see below, then go test -tags libantfly ./... in go/pkg/embedded). For a complete application, antfly-docs-knowledge-base builds a knowledge base over Antfly's docs on Lite: chunking and embedding, a GLiNER knowledge graph, hybrid search, graph traversal, and backups. Its model-free smoke command runs in zig build lite-test against the built libantfly.

A *embedded.DB is safe to share across goroutines, like *sql.DB. Searches and lookups run in parallel with each other and with writes; concurrent Batch calls on one handle queue rather than fail. Only one writer handle can have a file open at a time, including across processes. Set OpenOptions.BusyTimeout to wait for the other writer to close instead of failing immediately with embedded.Busy.

The other bindings mirror the Go API in each language's idiom:

# uv add antfly-embedded
import antfly_embedded

with antfly_embedded.create("py-app.aflite") as db:
    db.batch([antfly_embedded.WriteIntent(key="doc:py:1", value={"title": "embedded python lite"})], timestamp=1)
    print(db.status())

// cargo add antfly-embedded --features libantfly
use antfly_embedded::{Database, OpenOptions, WriteIntent};

let db = Database::create("rs-app.aflite", &OpenOptions::new())?;
db.batch(&[WriteIntent::put("doc:rs:1", r#"{"title":"embedded rust lite"}"#)], 1)?;
println!("{:?}", db.status()?);

// npm install @antfly/embedded
import { create } from "@antfly/embedded";

await using db = await create("ts-app.aflite");
await db.batch([{ key: "doc:ts:1", value: { title: "embedded typescript lite" } }], 1n);
console.log(await db.status());

Every handle is safe to share across threads (Python, Rust) or concurrent promises (TypeScript), with the same semantics as Go. Threads that call into Lite need at least 8 MiB of native stack; Go, Python, and the TypeScript binding already arrange that, and Rust callers should spawn threads with antfly_embedded::MIN_THREAD_STACK_SIZE. The same bindings also open a normal Antfly directory (for example embedded.OpenOptions{Storage: embedded.StorageDirectory} in Go), and a portable .afb backup restores or imports into either kind, so moving between a .aflite file and a directory is one backup and one restore. The shared conformance cases in zig/pkg/antfly-embedded/capi-conformance keep all four bindings behaviorally identical.

Building from Source#

The antfly-embedded release archive contains libantfly, include/antfly.h, pkg-config metadata, runtime files, and license notices. Python/npm packages bundle the library without CLI commands. Install antfly for local file and inference commands.

From the zig directory:

zig build release       # antfly CLI, library, and header
zig build antfly        # CLI only
zig build libantfly     # library and header
# Independent library composition, without server implementations:
zig build --build-file embedded.build.zig libantfly

Lite builds embed the local inference runtime, and status advertises it by default (-Dlite-local-inference-runtime=false hides it). The local database provider, including libantfly and Lite CLI enrichment, runs inference in-process on every backend, including Metal, CUDA, ONNX, and PJRT. It disables process isolation and does not use ANTFLY_INFERENCE_WORKER. A deadline or close cannot interrupt an active GPU driver call, and a driver fault can terminate the host process. Server hosts that enable process isolation re-execute antfly as a replaceable worker.

zig build wasm builds and installs the embedded database and inference WASM bundle, and zig build wasm-test runs its Node smoke test. zig build lite-test runs the Lite backend, CLI, C ABI, Go binding, embedded example, and packaging checks in one pass.

Maintenance#

antfly lite check app.aflite      # integrity
antfly lite compact app.aflite    # compact index work and reclaim space
antfly lite vacuum app.aflite     # file-level vacuum only

For a production application, pair regular .afb backups with check.

Use Agent Skills#

Everything above is also encoded in the Antfly skill, so a coding agent can execute this guide for you:

npx skills add antflydb/antfly-skills

Then prompt it with the outcome, for example "Embed Antfly Lite in my Go application with a single-file database and a full-text index", and use this page to judge the result.

Next Steps#

Quickstart: the same queries against a normal standalone instance
Document Engine: how mappings, full-text, vector, and graph indexes compose under one query, in Lite and everywhere else
Backup and Restore: the .afb format and restore options on a normal instance