Decisions
Standalone decisions use POST /decisions (/ai/v1/decisions on the standalone
server). Named question and answer arrays, input, choices, levels and
predicate probability follow the OpenAI Decisions conventions.
Antfly adds embedding similarity answers, multi-choice selection and text
batches. Supported hosted providers also accept multimodal evidence; see the
gateway decision contract
for provider and method restrictions. Native trained/embedding decisions and the
browser surface retain their text restrictions. Choice values are strings and
every question needs a unique name. See the decision schema.
/extract retains entities, relations, attributes, structured records and
ordinary classification. Standalone GLiDE, Laya, OpenDecider and EmbeddingGemma
workflows use /decisions. The unreleased extraction decision output and
Boolean and ordinal decision modes have been removed.
For the v0.3.0 development source documented here, standalone public inference
requires an explicit opt-in before calling /ai/v1. Save antfly.json:
{"public_inference": {"enabled": true}}
antfly standalone --config antfly.json
There is no enablement CLI flag. Without this setting, public execution returns
HTTP 403 even when models are installed. See the
public execution policy.
This prerequisite applies to database API nodes on this source version; the
independent antfly inference run listener and older stable releases have their
own policies.
Questions and answers
{
"model": "decision-model",
"input": "Please refund the duplicate charge before tomorrow.",
"questions": [
{"name": "route", "type": "choice", "instructions": "Which team?", "choices": [{"value": "billing", "description": "Charges and refunds"}, {"value": "support", "description": "Product troubleshooting"}]},
{"name": "urgency", "type": "score", "instructions": "How urgent?", "levels": [{"label": "routine"}, {"label": "soon"}, {"label": "immediate"}]},
{"name": "refund", "type": "predicate", "instructions": "The input requests a refund."}
]
}
Trained answers include name, type, decision_method: "typed" and:
| Type | Fields |
|---|---|
choice | choice, probabilities: [{value, probability}], confidence, confidence_method |
score | score, probabilities: [{value, label, probability}], confidence, confidence_method |
predicate | probability, optional confidence diagnostics |
Scores are expected zero-based ordinal indices and can be fractional. A
predicate estimates whether the statement is true. Confidence is a model
diagnostic, not the probability that a chosen label is correct. Models with
an action head retain act_probability. Applications apply acceptance policy
and validate arguments before executing an action.
A single response contains model, an answers array in question order, and
aggregate token usage. For batches, replace input with
inputs: [{"id": "first", "input": "..."}, {"input": "..."}]. The response uses
data: [{input_index, id?, answers}] instead of top-level answers. Questions
are shared; IDs and order are preserved. Limits: 128 inputs, 64 questions,
2–64 choices or levels, 1 MiB per input and 16 MiB per request. Model and
executor limits can be smaller.
Shared context batches
Supporting Antfly servers accept the explicit context_version: 1 transport
extension. Send immutable context strings once and reference their zero-based
indices from individual inputs:
{
"model": "decision-model",
"context_version": 1,
"contexts": ["File: retry.py\nCONTEXT: def retry(): ..."],
"inputs": [
{"id": "line-1", "input": "TARGET: retry_count += 1", "context_index": 0},
{"id": "line-2", "input": "TARGET: return response", "context_index": 0}
],
"questions": [{"name": "relevant", "type": "predicate", "instructions": "Does TARGET implement retry enforcement? Use CONTEXT only to interpret TARGET."}]
}
Each referenced item is evaluated with exactly context + "\n" + input.
Unreferenced items keep their original input. Answer IDs, indices, and question
semantics are unchanged. Unknown versions, missing context arrays, invalid
indices, and references without a version fail validation. Context arrays contain
at most 128 nonempty UTF-8 strings; effective inputs remain limited to 1 MiB each
and their combined expanded size is limited to 16 MiB before model admission.
Model limits may reject smaller requests. Shared transport reduces request bytes;
it does not imply shared model computation or lower billed token usage.
antfly-grep --decision-context shared-v1 opts into this format. Its default
legacy transport remains compatible with existing endpoints.
Embedding similarity and multi-choice
EmbeddingGemma 2 advertises embedding_similarity and
supports choice and multi_choice. Answers include
decision_method: "embedding_similarity", similarity_metric: "cosine" and
similarities: [{value, similarity}]. Raw cosine values range from -1 to 1;
these answers have no probability or confidence fields. Cosine is fixed because
thresholds, margins and calibration depend on its units. Another metric would
require a separately qualified contract.
Request-wide embedding_options supplies only geometry defaults: task_type
(CLUSTERING or CLASSIFICATION) and dimensions (768, 512, 256 or 128).
Each question has its own acceptance embedding_options: min_similarity,
min_margin or calibration_id. The embedding endpoint also accepts
task_type; decision routing restricts the supported prompt profiles.
calibration_id is a public identifier for a qualified artifact at
<model-dir>/calibrations/<id>.json. The runtime verifies model identity,
prompt profile, dimensions, renderer, mode and that question's prototypes.
Calibration is mutually exclusive with manual thresholds. See the model guide
for fitting; thresholds must be evaluated on application data.
For single choice, min_margin is the top-two cosine gap. Ties abstain:
choice is null and status is abstained. Multi-choice requires
similarity_thresholds (one cosine threshold or a complete value-to-threshold
map), unless calibration supplies them. Its min_margin is the nearest absolute
distance to any label's threshold. The answer includes effective thresholds and
choices: selected means a nonempty accepted set, empty is a valid empty
set, and abstained means the boundary margin failed. These states are distinct.
HTTP, CLI and bindings
curl --fail-with-body http://127.0.0.1:8080/ai/v1/decisions \
-H 'Content-Type: application/json' --data-binary @decision.json
antfly inference decisions ./models --request decision.json --backend metal
Repeat --request to run several files against one loaded model; the CLI prints
one JSON response per line. --repeat N re-runs the files N times, interleaved,
and prints each file's median and minimum latency on stderr.
The inference listener serves /decisions directly (default port 8090).
Existing binding methods consume the same JSON: C antfly_inference_decide_json,
Rust Inference::decide, Go Inference.Decide, Python Inference.decide,
TypeScript Inference.decide / decideRaw. HTTP methods remain Python
AntflyClient.decide and TypeScript InferenceClient.decide.
response = client.decide(request)
route = next(answer for answer in response.answers if answer.name == "route")
print(route.choice)
const response = await client.decide(request);
const route = response.answers?.find(answer => answer.name === "route");
if (route?.type === "choice") console.log(route.choice);
Reuse handles for cached execution. Bindings preserve error bodies and retry metadata. Free C output buffers on success and failure and close handles; see C API ownership.
SQL providers
deciders:
triage:
provider: antfly
model: decision-model
max_rows: 10000
max_input_tokens: 1000000
batch_size: 32
SQL references the configured name. Omitting url uses linked inference when
available. For HTTP use http://127.0.0.1:8090, or the standalone base including
/ai/v1; the provider appends /decisions. OpenAI uses /decisions; Jev's
private adapter retains its upstream wire format. Responses normalize to the
same public answer arrays.
SELECT ai_decide('Please refund my charge.',
'[{"name":"refund","type":"predicate","instructions":"The input requests a refund."}]'::jsonb,
'triage');
SELECT ai_choice('Duplicate charge', 'Which team?',
'{"billing":"Payments and refunds","support":"Product troubleshooting"}'::jsonb, 'triage');
SELECT ai_score('Respond before tomorrow.', 'How urgent?',
'["Routine","Soon","Immediate"]'::jsonb, 'triage');
SELECT ai_probability('Please refund my charge.', 'The input requests a refund.', 'triage');
ai_decide accepts named question arrays and returns the full response.
Convenience functions return a choice, expected score or true probability.
Embedding deciders use decision_method: embedding_similarity; trusted config
can supply geometry and acceptance defaults. Serialization moves acceptance
onto each question; question policies override those defaults. ai_choice
returns SQL NULL on abstention, and ai_decide supports multi-choice. Scores and
predicates require trained deciders. A database handle alone does not configure
SQL providers.
Validate quality, thresholds and resource limits on your workload before deployment. Preparation and limits are in GLiDE, Laya and EmbeddingGemma 2.
Decider cascades
A cascade is a named decider that asks a cheap decider first and sends only
uncertain inputs to a stronger one. Configure it under deciders with
decision_method: cascade; each stage is an inline typed or generated decider.
deciders:
same-entity:
decision_method: cascade
cascade:
stages:
- name: fast
decider: {provider: jev, model: jev-latest}
- name: strong
decider:
decision_method: generated
probabilities: verdict_only
generator: {provider: anthropic, model: your-strong-model}
when:
question: same
choice: same
probability_between: [0.3, 0.9]
budget: {max_model_calls: 2, deadline_ms: 10000}
Select it like any other decider: "model": "deciders/same-entity" on
/ai/v1/decisions, the decider name in SQL, or the inline configuration in a
resolver adjudicate policy or decision enrichment.
- The first stage answers every input. A later stage runs only for inputs whose
previous answer falls inside its inclusive
probability_betweenband, and those inputs share one call.questionlimits the band to one question, andchoicemeasures that value's probability instead of the selected value's. - The final answer is the last answering stage's. Usage is summed over the
stages that ran for the input. Shared generated-batch usage is apportioned
evenly across each call's inputs, with integer remainders assigned in input
order, and marked
usage.estimated: true. Request totals remain measured. If reported usage exceeds a stage's token limit, every completed call is charged before the stage fails and future windows stop.keep_previousretains those paid costs in the failed-stage trace and final usage; inputs that were not submitted incur no model usage. - Only choice and predicate questions are accepted, because bands read
probabilities. Embedding-similarity deciders and nested cascades cannot be
stages, and a band cannot follow a
verdict_onlygenerated stage. This also applies to dynamically returned proxy answers: a verdict's0/1assertion cannot drive a confidence band. An unmeasurable result follows its producing stage's failure policy, retaining paid usage and completed siblings. A valid veto stops before any later band's evidence is inspected. - Escalation applies to successful answers. It is not failover: a stage error
ends the input unless that stage sets
on_failure: keep_previous, which keeps the previous answer and records the error. The first stage defaults tofail; explicitly settingon_failure: failis equivalent to omission, whilekeep_previousrequires a preceding stage. A later stage's operational failure does not discard confident siblings that already completed an earlier stage. budget.max_model_calls(at most 8, at least the stage count) bounds stage invocations;budget.deadline_msnarrows the caller's deadline.- Each stage keeps its own rate limits, row and token budgets, and media limits across batches in the same statement. Set row, token and batch limits on the stage deciders. The cascade admits images and message evidence only when every stage does. Question and choice selectors are validated against the request before any stage runs. Native model discovery runs only when a stage is reached, under its deadline and failure policy.
Set require_agreement: true on a later stage to require its vote to match
that of the preceding stage before accepting the result:
- name: strong
decider:
decision_method: generated
generator: {provider: openai, model: your-strong-model}
when: {question: same, choice: same, probability_between: [0.3, 1.0]}
require_agreement: true
This band sends every positive candidate, including confident positives,
for confirmation. A narrower uncertainty band leaves confident earlier answers
unchecked. when.choice selects the probability used for escalation; agreement
compares selected choice strings. Predicate votes must be strictly on the same
side of 0.5; a tie at 0.5 does not agree. when.question scopes the comparison
to that question; without it, all questions must agree.
Disagreement stops that input immediately. Later stages cannot override the
veto, and other inputs continue independently. Raw final answers and accumulated
usage remain available through ai_decide and the Decisions API; SQL
ai_choice, ai_score, and ai_probability return NULL. Predicate convenience
clients return DecisionAbstained. Resolver adjudication remains REVIEW even
with on_failure: mint, because disagreement is an abstention rather than an
operational failure.
Guarded stage traces and the aggregate trace contain agreement_status:
agreed, disagreed, or unverified. The derived agreed boolean appears for
completed comparisons, and sorted disagreed_questions appears only for
disagreement. A guarded stage failing with keep_previous retains the earlier
answer but reports unverified and abstains. No agreement fields appear when
neither a local nor an upstream check ran. A successful check remains recorded
if a later unguarded stage runs; guard every later stage whose answer must also
be confirmed.
An upstream cascade's disagreed or unverified result also stops an outer
cascade, including at its first or an unguarded stage. Stage receipts preserve a
bounded upstream_agreement summary with its reported status, disagreement
question names, and available cascade identity. They do not recursively embed
upstream traces or claim to verify the upstream definition. A local required
comparison is unverified when its upstream abstains; the aggregate retains the
upstream veto. Resolver policies honor this abstention even when they use a
plain typed proxy and on_failure: mint. When an upstream response supplies a
full trace, its bounded stage structure, aggregate status, revision metadata,
and final answer must be coherent. Metadata-only legacy summaries remain
supported; upstream policy definitions are not reconstructed locally.
A performed local comparison records sorted agreement_questions on its stage.
Consumers recompute that comparison from the preceding and current answers,
including strict predicate tie handling. Every successful stage also passes a
shared, bounded vote validator before its selected answer can authorize
agreement: typed choices must maximize normalized finite distributions, generated
choices must match their raw sampled answer, and categorical verdicts must carry
consistent 0/1 evidence. Validation does not rebuild producer plans per result. Older full traces without scope remain
usable when the answer evidence proves agreement; ambiguous historical scoped
agreement abstains as unverified. Reported disagreement must identify actual
differing votes. A flat dependency projection cannot coexist with a full trace
and hide its generated revisions or definition identity. Projections remain on
stage receipts and are not copied onto the public response root.
For provider: antfly, the default typed method selects the Decisions
transport. A remote named decider may return typed or generated answers,
including a sampled choice that differs from its distribution's argmax. Proxy
consumers validate each answer's reported method and probability evidence before
honoring agreement. Generated token probabilities and categorical verdicts keep
their original meaning; embedding-similarity responses still require explicit
configuration. A stage receipt records its configured execution method; each
answer retains the method that actually produced it.
Generated choice votes retain the sampled structured answer, which need not be the highest-probability alternative. Resolver thresholds apply to that selected vote; probability mass on another choice cannot accept or reject the pair.
Disabled guards are omitted from forwarded and persisted configuration, so unguarded cascades remain readable at epoch 30. Enabled guards require epoch 31. SDK constructors omit the optional guard unless explicitly configured.
Agreement records matching votes; it does not establish model independence or calibrated confidence. Use suitable independent evidence and evaluate the resulting policy on representative data.
Responses carry a cascade trace per input (top-level for single input, on each
batch row otherwise):
{"cascade": {"decider": "same-entity", "definition_sha256": "…", "answered_by": "strong",
"stages": [
{"name": "fast", "decision_method": "typed", "model": "jev-latest", "answers": [...], "usage": {...}, "escalated": true},
{"name": "strong", "decision_method": "generated", "model": "your-strong-model", "answers": [...], "usage": {...}, "escalated": false}]}}
definition_sha256 identifies the cascade definition without credentials.
Anthropic stages require probabilities: verdict_only, since they do not
provide token logprobs. Such stages can appear only last. Resolver probability
thresholds still apply to the probabilistic stages; a terminal verdict-only
stage must choose categorically, and its reported 0/1 distribution is not a
calibrated confidence estimate.
Stage traces also retain a flat upstream_revisions manifest, bounded to 32
entries and eight stage path levels. It records every successful generated
model revision and the upstream cascade definitions, including dependencies
before a typed final answer. A cascade definition remains a versioned dependency
even when every stage is typed or no provider reports a model revision.
New projections use upstream_revisions_version: 2 with an explicit
upstream_revisions_complete boolean. The node array retains actual model and
cascade dependencies; false carries uncertainty without consuming any of its
32 slots. Version-1 projections remain readable, with legacy unverified entries
folded into uncertainty. Only a complete empty projection establishes that no
versioned dependencies were reported. Older projections may
have dropped definition-only subtrees even while retaining other model nodes.
Their old hashes (including the preceding synthetic-marker format) are recognized
for migration, but cannot establish completeness or authorize identity verdict
reuse by themselves. A validated historical veto can still retract MATCH to REVIEW
offline at the full 32-dependency limit. Unavailable verification preserves the committed artifact;
explicit configuration/source invalidation permits fresh inference. Resolver replay requires an
authoritative verifier for composite routes and preserves stored receipts when
that verification is unavailable. Stored successful responses are reclassified
under the current policy without model inference; an upstream veto retracts a
historical accepted verdict to REVIEW and restores the mention's own endpoint.
Ordinary low-confidence abstentions still verify model revisions. Veto receipts
also refresh when authoritative verification succeeds; an unavailable verifier
can retain the recorded veto without another inference call.
Resolution artifacts carry one optional adjudication_execution_hash for their
shared policy/source binding, independent of model revisions. This lets changed
inputs invalidate old failure receipts before obsolete dependencies need
verification. Legacy receipts retain verified migration; a recorded changed
mention can also prove invalidation. Ambiguous legacy bindings remain preserved
until they can be verified or explicitly invalidated by a new configuration.
Revision discovery and inference share one monotonic deadline, narrowed by any
cascade deadline. Each lookup receives only the remaining whole milliseconds;
discovery exhaustion preserves committed evidence. Executor requests also carry
the shared budget; serial and batch completion checks reject late results with
Timeout before publishing any identity decisions, including under
on_failure: mint. Malformed dependency metadata fails only its own batch row,
without discarding healthy sibling results. A validated recorded veto
can still retract an older accepted verdict to REVIEW.
Stage traces include model_revision when the provider reports an immutable
revision. Resolver adjudication verifies the definition, escalation decisions,
stage order, and agreement between the trace and the final answer.
Probabilities from different models are not calibrated against each other, so
set each band on labelled examples for the models involved and measure how
often inputs escalate. The trace has what an evaluation needs: the answering
stage, whether each answer escalated, and per-stage usage.