Index PDFs and Scanned Documents
A table that extracts text from PDFs, reads scanned pages and images with OCR, and serves the result to full-text search
The Result
You write a document whose url points to a PDF or an image, and Antfly extracts each page. Pages with a usable text layer keep their native text; scanned pages, photos, and images are read by an OCR model. The text is chunked into full-text search, and each hit names the document and the page that matched:
curl -X POST http://127.0.0.1:8080/db/v1/tables/scans/query \
-H 'Content-Type: application/json' \
-d '{"full_text_search": {"query": "grid"}, "fields": ["_id"], "limit": 3}'
{
"responses": [{
"hits": {
"total": {"value": 1, "relation": "exact"},
"hits": [{
"_id": "scan-1",
"_score": 2.17,
"hierarchy": {
"level": "source",
"parent_doc_key": "scan-1",
"matched_artifact": {"name": "document_chunks_v1", "kind": "chunk", "chunk_id": 0, "unit_id": "page:000001"}
}
}]
}
}]
}
Before You Start
Antfly running in standalone mode:
curl -fsS http://127.0.0.1:8080/db/v1/status
Build It
1. Install an OCR Model
LightOnOCR-3 transcribes page images, including scans and photos, into Markdown with tables and formulas. Pull the 0.8B checkpoint:
antfly inference pull lightonai/LightOnOCR-3-0.8B:safetensors@36d8636925327c671bd8e479c517a004f7566bd9
The 4B checkpoint, lightonai/LightOnOCR-3-4B, is more accurate on dense or degraded pages and needs about 9 GB instead of 1.7 GB.
The pull reference pins the installed weights. Requests use a selector advertised by `GET /ai/v1/models`: the bare repository name below selects the sole installed artifact (or the configured default). Discovery also advertises variant-qualified and immutable revision selectors for managed installations. Use the advertised immutable selector when the enrichment must keep using that exact artifact; unknown revisions fail instead of silently selecting another checkpoint. Manually copied models without managed installation receipts retain their literal local identifiers. Pull the artifact through the registry before requesting a verified format or revision; a directory name alone does not prove those pins.
2. Create the Table
curl -fsS -X POST http://127.0.0.1:8080/db/v1/tables/scans \
-H 'Content-Type: application/json' -d '{"num_shards": 1}'
Add a document_extraction enrichment that reads the url field. The ocr block names the reader:
curl -fsS -X PUT http://127.0.0.1:8080/db/v1/tables/scans/artifacts/document_units_v1/enrichment \
-H 'Content-Type: application/json' \
-d '{
"kind": "asset",
"field": "url",
"content_type": "application/json",
"producer_json": "{\"type\":\"document_extraction\",\"config\":{\"ocr\":{\"reader\":{\"provider\":\"antfly\",\"model\":\"lightonai/LightOnOCR-3-0.8B\"}}}}"
}'
Then chunk the extracted text into the table's full-text index:
curl -fsS -X PUT http://127.0.0.1:8080/db/v1/tables/scans/artifacts/document_chunks_v1/enrichment \
-H 'Content-Type: application/json' \
-d '{
"kind": "chunk",
"source_artifact_name": "document_units_v1",
"field": "text",
"chunk_size": 256,
"full_text_index": true
}'
Omit Page-Number Scraps
For document corpora where page labels or punctuation-only pages produce irrelevant vector matches, use a fixed chunker with an explicit minimum:
{
"name": "document_chunks_v1",
"kind": "chunk",
"source_artifact_name": "document_units_v1",
"field": "text",
"chunker": {
"provider": "antfly",
"store_chunks": true,
"text": {"target_tokens": 256, "overlap_tokens": 32, "min_chars": 16}
},
"full_text_index": true
}
text.min_chars counts non-whitespace Unicode scalar values in each completed
text chunk. Chunks below the threshold are omitted before storage, full-text
indexing, and downstream embedding. Source text remains in the extraction asset,
and retained chunks keep their original source offsets. This avoids an
English-word heuristic, so the same threshold can be tuned for other scripts.
It is a length rule, not a semantic relevance or punctuation detector.
The default is 0 (disabled), preserving short captions, equations, and
identifiers. Choose the threshold for your corpus. It applies to fixed text
chunking, including /ai/v1/chunk, and does not discard visual PDF pages or
model-aware audio/image windows. After changing the policy or upgrading PDF
extraction, reprocess the affected documents so existing chunks and vectors are
replaced. A document whose chunks all fall below the threshold publishes an
empty inventory and retires its previous full-text entries and chunk vectors.
Native PDF Text
Native PDF extraction does not use a model or OCR. It decodes the PDF's text
operators and honors /ActualText replacements, including marked content in
Form XObjects and structure elements linked through ParentTree and MCID.
Shared structure replacements emit once and retain the union of their paint
evidence without removing unrelated interleaved text. Configure native extraction separately from OCR:
{
"native_text": {
"policy": "page",
"quality": {
"min_chars": 50,
"max_replacement_ratio": 0.05
}
}
}
policy selects which source text is eligible:
| Policy | Behavior |
|---|---|
page (default) | Excludes runs wholly outside the page or a known clip; retains invisible OCR text layers. |
visible | Also excludes nonpainting or fully transparent text. This is not pixel-level occlusion detection. |
source | Retains unfiltered source runs for auditing, including exterior and invisible text. |
All policies honor /ActualText; none performs spelling correction or reconstructs
a missing text layer. A rotated label does not disable geometric ordering for
unrelated upright text. Ambiguous text, tables, and overlapping transcription
tracks retain conservative authored ordering.
Every PDF unit records native_text_quality and native_text_diagnostics, even
without an ocr block. Quality flags describe short text, sampled character/word
corruption, and replacement characters; they are heuristics, not an accuracy
score. Diagnostics record the selected policy, ActualText replacement count,
exterior/invisible glyph runs and semantic spans, unpositioned semantic spans,
alignment and ordering limitations, and nontext or
annotation evidence. extraction_complete describes decoding, not correctness.
actual_text_incomplete reports malformed or unclosed inline replacements,
or malformed, unsupported, or resource-limited structure replacement lookup.
Native glyph text is retained when a replacement cannot be applied;
cross-page replacements and object-reference (OBJR) content are reported as
partial coverage rather than assigned invented page text or geometry.
Optional null metadata behaves like an omitted value; an empty /ActualText
string intentionally deletes the covered source text.
semantic_coverage is partial when unresolved coverage is known, unknown
when extraction is incomplete, or text_only otherwise; it never certifies
headings, tables, chart interpretation, or complete visual understanding.
text_regions include page-local UTF-8 byte spans, page-space bounding boxes,
effective font_size, and optional bold/italic font evidence. text_blocks
group compatible runs into geometric lines in emitted reading order, with the
same style evidence and explicit reading_order. They are not inferred
paragraphs or table cells. OCR replacement clears native blocks rather than
leaving byte offsets pointing into different text; native quality and diagnostics
remain available as pre-OCR provenance.
native_text.quality controls native assessment and automatic OCR routing;
ocr.quality assesses OCR output independently. Both use the same text-quality
options: min_chars and min_line_chars count non-whitespace Unicode characters,
max_replacement_ratio limits replacement characters, and max_font_corruption
sets the corrupted-line score threshold. Defaults work without any quality
configuration. Images go directly to OCR when a reader is configured; they do
not require a native text layer.
After changing extraction settings, reprocess affected documents so stored text, geometry, chunks, and vectors use the new policy.
For a source checkout, build and inspect only the PDF component:
cd zig
ANTFLY_ZIG_MAX_RSS=12884901888 python3 tools/run_bounded_zig_build.py --zig zig -- \
build lib-pdf-bench lib-pdf-unit-test -Doptimize=Debug -Dpdf-optimize=Debug -j1
./zig-out/bin/lib-pdf-bench dump-analysis input.pdf analysis.json page
./zig-out/bin/lib-pdf-bench dump-text input.pdf text.txt source
These targets do not build the Antfly server. dump-analysis includes emitted
text, diagnostics, blocks, original positioned glyph runs, and semantic_spans.
Each /ActualText replacement is stored once with its source run range, output
span, and policy-selected bounds. Glyph text and raw_text retain their source
values; covered glyphs refer to the replacement by semantic_span_index.
Unknown paint bounds are retained conservatively and omitted from geometry
metadata rather than assigned invented coordinates. Its optional final
argument selects source, page, or visible (default page). The broader
lib-pdf-test target also includes OCR integration and is not PDF-only.
Scanned PDFs with an existing invisible OCR text layer retain that layer's positions for word and line reconstruction. Within an overlapping region, extraction retains alternate transcriptions and their source spans, including independent invisible OCR copies. Words belonging to an overlapping text object stay with that authored transcription, even when their positions differ. Ambiguous overlap layouts fall back to authored text-object boundaries. Complementary text without overlaps still follows geometric reading order. Unrelated headings, footers, and columns keep their existing reading order. Antfly does not paint invisible text over the scanned image.
3. Write a Scanned Document
url can be a remote URL or an inline data URI. This writes a local PDF inline:
curl -fsS -X POST http://127.0.0.1:8080/db/v1/tables/scans/batch \
-H 'Content-Type: application/json' \
-d "{\"inserts\": {\"scan-1\": {\"url\": \"data:application/pdf;base64,$(base64 < scan.pdf | tr -d '\n')\"}}, \"sync_level\": \"write\"}"
PNG and JPEG images work the same way with a data:image/png;base64,... or data:image/jpeg;base64,... URL.
4. Check the Extracted Pages
Each page becomes a unit under its source document. List them:
curl -fsS -X POST http://127.0.0.1:8080/db/v1/tables/scans/query \
-H 'Content-Type: application/json' \
-d '{
"fields": ["unit_type", "extraction_status", "extraction_warning", "text", "native_text_quality", "native_text_diagnostics", "text_blocks", "provenance.page_number"],
"hierarchy": {"children": {"parent": {"level": "source", "id": "scan-1"}, "level": "unit"}},
"order_by": [{"field": "_hierarchy.position"}]
}'
Each unit's text holds the page's native text or the reader's transcription, and extraction_status reports where it is: pending_ocr while the reader runs, completed when the text is ready, failed_ocr if the reader failed, and ocr_not_configured if the page needed OCR but the enrichment has no ocr block. LightOnOCR returns Markdown, so a scanned table comes back as a table and a figure as a short description.
completed is a pipeline status, not a text-accuracy guarantee. Inspect
native_text_quality and native_text_diagnostics when deciding whether native
PDF output needs review or OCR.
5. Search the Text
Query the table as shown in The Result. A hit is the source document, and hierarchy.matched_artifact.unit_id names the page whose chunk matched; read that page's text with the query in step 4. Keep fields narrow, because the source document includes its url, which for an inline upload is the whole file. OCR output is chunked and indexed like any other text, so it also works with hybrid search once you add an embeddings index over document_chunks_v1.
OCR Options
The ocr block takes:
| Field | Meaning |
|---|---|
reader | A reader provider config. Every provider except apple names a model. |
generator | Instead of reader, a vision-capable generator, which receives a transcription instruction. |
mode | auto (default) reads images and the PDF pages whose native text fails native_text.quality. always reads every page. |
prompt | Overrides the prompt. Readers run their own OCR task when it is omitted. |
render | How PDF pages are rasterized: dpi (default 150), resolution (requested_dpi or model_input), max_pixels, max_dimension. |
quality | Thresholds used to assess OCR output, independently of native_text.quality. min_chars counts non-whitespace Unicode characters (default 50), matching text chunking. |
Without an ocr block, document_extraction extracts native text only: born-digital PDFs, Office documents, HTML, email, and plain text, with no model. Pages and images that would need OCR keep whatever native text they have and report extraction_status: "ocr_not_configured", so a scan never silently indexes as empty.
Other Readers
Any reader or vision-capable generator can take LightOnOCR's place:
- Apple Vision on macOS, with no model to install:
{"reader": {"provider": "apple"}}. See Apple. - Smaller local readers:
antflydb/Florence-2-baseormonkt/paddleocr-onnxfor a small CPU footprint on clean printed text. - A vision-capable generator, local or hosted, such as Gemma 4 or a model from OpenAI or Google:
{"generator": {"provider": "...", "model": "..."}}.
Tradeoffs
The reader decides accuracy, cost, and speed, and mode decides how often you pay for it. LightOnOCR-3 0.8B is the default choice: it keeps table and reading-order structure, but it generates text token by token, so a page costs far more than native extraction. Handwriting, photos, and degraded scans go through the same OCR path; when they read poorly, switch to the 4B checkpoint, Apple Vision, or a hosted vision model rather than changing anything else in the pipeline. mode: "auto" sends only the pages that need it to the reader, which keeps born-digital PDFs cheap; use always when a document's text layer is present but untrustworthy, for example a scanner's own OCR layer. Raise render.dpi for small print, at the cost of memory and time per page. A hosted generator moves the cost to per-request pricing and sends page images to that provider.
If the reader fails, for example because the model is not installed, the page reports extraction_status: "failed_ocr". Fix the cause and reprocess the artifact to retry.
On a server with an enrichment runtime, document-artifact reprocessing durably submits a target generation and returns before extraction runs. The worker retains that request across cold starts, transient provider failures, and owner restart. A successful HTTP response acknowledges submission; inspect the artifact's generation, merge status, and OCR failure counts to observe completion. Embedded databases without an enrichment runtime execute extraction inline.
For a collection of failed documents, use the existing table repair jobs rather than holding a synchronous repair request open through remote OCR:
curl -X POST "$ANTFLY_URL/db/v1/tables/docs/repair/jobs" \
-H "Authorization: Bearer $ANTFLY_API_KEY" \
-H 'Content-Type: application/json' \
-d '{"target":"artifact","kind":"asset","index":"page_text","limit":16,"advance":false}'
Retain the returned job_id, POST to /repair/jobs/{job_id}/advance, and GET
/repair/jobs/{job_id} to inspect progress. Advance queued jobs until terminal;
concurrent advances share the job's execution lease. Native document-extraction
and embedding repairs submit durable work in each pass, so a pass may report
in_progress while provider work continues. Generic asset and chunk producers
retain their synchronous execution semantics. A later pass observes successful
publication or a failed attempt; a fresh job explicitly retries failed work.
The job's work counters accumulate across passes. Its in_progress and
unresolved counts describe the latest bounded pass, so polling the same queued
source does not inflate them. Earlier failures remain in the cumulative failure
counts and debt_remaining; remaining_failures summarizes current document debt.
Publication retires only older matching consumer debt, after a healthy extraction manifest and its index updates have reached that consumer. A partial or failed manifest retains repair debt, even if older searchable text is still present.
Read extraction_warning for the cause. ReadOutputTruncated means the reader
reached its token budget before completing the page; the
LightOn guide
shows an explicit dense-page budget and explains memory admission failures.
Use Agent Skills
Everything above is also encoded in the Antfly skill, so a coding agent can execute this guide for you:
npx skills add antflydb/antfly-skills
Then prompt it with the outcome, for example "Index the scanned PDFs in my contracts bucket with OCR and make them searchable", and use this page to judge the result.
Next Steps
- Artifact-Backed Indexes to search whole documents and their extracted chunks in one index.
- LightOnOCR-3 for the model's request options and grounding output.
- Multimodal Embeddings to search images by what they show rather than the text they contain.