Compare structured-output contracts across LLM APIs
LLM Structured Output Compared — JSON Mode and JSON Schema by API
JSON mode, strict JSON Schema, tool-argument schemas, and typed SDK helpers are different contracts. Compare the exact provider, host, model, surface, request field, schema subset, refusal and finish signals, and validator version. A syntactically valid response is not schema-conformant until validated; unsupported or changed surfaces remain conditional or Unavailable. Verified 2026-09-02.
Verified 2026-09-02 — source
Compare JSON mode, strict schemas, tool arguments, streaming, refusal, truncation, and validator-backed conformance. Qualitative demand: dedicated cross-provider structured-output guides were reviewed 2026-09-02; exact US monthly volume is unavailable.
Batch 58 · server-rendered evidence boards · verified 2026-09-02
Intent answer: JSON mode, strict JSON Schema, tool-argument schemas, and typed SDK helpers are different contracts. Compare the exact provider, host, model, surface, request field, schema subset, refusal and finish signals, and validator version. A syntactically valid response is not schema-conformant until validated; unsupported or changed surfaces remain conditional or Unavailable. Verified 2026-09-02.
Demand evidence: Qualitative demand: dedicated cross-provider structured-output guides were reviewed 2026-09-02; exact US monthly volume is unavailable.
Scope boundary: Compare JSON mode, strict schemas, tool arguments, streaming, refusal, truncation, and validator-backed conformance. Exact provider, seller, serving host, account/tier, endpoint, model/snapshot, region, feature, and evidence identity are required; unresolved joins render Unavailable.
Structured-output wire crosswalk
Deterministic formula / rule: native conformance = documented surface + exact support + request/response fields + validator pass; valid JSON alone is insufficient.
Boundary: Owns cross-provider wire and schema protocol; semantic extraction accuracy remains with task and test owners.
| Frozen scenario / field ID | Exact identity and evidence fields | Result | State |
|---|---|---|---|
batch58-structured-output-m1-r1plain JSON mode | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=plain JSON mode; provider/host/model/surface; request field; fragment hash; schema location; strictness; response/refusal/finish; source/date; state; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — frozen structured-output fixture requires an exact source, identity, and result receipt | UNTESTED — module 1 rule is reproducible but no production observation is claimed |
batch58-structured-output-m1-r2strict response schema | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=strict response schema; provider/host/model/surface; request field; fragment hash; schema location; strictness; response/refusal/finish; source/date; state; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — frozen structured-output fixture requires an exact source, identity, and result receipt | UNTESTED — module 1 rule is reproducible but no production observation is claimed |
batch58-structured-output-m1-r3tool-argument schema | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=tool-argument schema; provider/host/model/surface; request field; fragment hash; schema location; strictness; response/refusal/finish; source/date; state; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — frozen structured-output fixture requires an exact source, identity, and result receipt | UNTESTED — module 1 rule is reproducible but no production observation is claimed |
batch58-structured-output-m1-r4typed SDK helper | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=typed SDK helper; provider/host/model/surface; request field; fragment hash; schema location; strictness; response/refusal/finish; source/date; state; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — frozen structured-output fixture requires an exact source, identity, and result receipt | UNTESTED — module 1 rule is reproducible but no production observation is claimed |
batch58-structured-output-m1-r5streaming structured response | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=streaming structured response; provider/host/model/surface; request field; fragment hash; schema location; strictness; response/refusal/finish; source/date; state; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — frozen structured-output fixture requires an exact source, identity, and result receipt | UNTESTED — module 1 rule is reproducible but no production observation is claimed |
batch58-structured-output-m1-r6unsupported-model | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=unsupported-model; provider/host/model/surface; request field; fragment hash; schema location; strictness; response/refusal/finish; source/date; state; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — exact structured-output evidence join is not closed for “unsupported-model” | FAIL CLOSED — manual, probe, or source evidence required |
First-party citation: OpenAI Responses API reference. Verified 2026-09-02; missing or conflicting joins fail closed.
JSON Schema subset conformance canary
Deterministic formula / rule: schema pass = canonical schema hash + provider-normalized schema + compile/request pass + syntax pass + schema validator pass.
Boundary: Semantic correctness is deliberately out of scope; no result transfers across model or surface revisions.
| Frozen scenario / field ID | Exact identity and evidence fields | Result | State |
|---|---|---|---|
batch58-structured-output-m2-r1required-and-enum | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=required-and-enum; canonical/provider schema hashes; compile/request result; output hash; syntax; validator/version; schema result; semantic result out of scope; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — frozen structured-output fixture requires an exact source, identity, and result receipt | UNTESTED — module 2 rule is reproducible but no production observation is claimed |
batch58-structured-output-m2-r2nested object/array | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=nested object/array; canonical/provider schema hashes; compile/request result; output hash; syntax; validator/version; schema result; semantic result out of scope; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — frozen structured-output fixture requires an exact source, identity, and result receipt | UNTESTED — module 2 rule is reproducible but no production observation is claimed |
batch58-structured-output-m2-r3additionalProperties=false | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=additionalProperties=false; canonical/provider schema hashes; compile/request result; output hash; syntax; validator/version; schema result; semantic result out of scope; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — frozen structured-output fixture requires an exact source, identity, and result receipt | UNTESTED — module 2 rule is reproducible but no production observation is claimed |
batch58-structured-output-m2-r4union/anyOf | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=union/anyOf; canonical/provider schema hashes; compile/request result; output hash; syntax; validator/version; schema result; semantic result out of scope; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — frozen structured-output fixture requires an exact source, identity, and result receipt | UNTESTED — module 2 rule is reproducible but no production observation is claimed |
batch58-structured-output-m2-r5recursive reference | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=recursive reference; canonical/provider schema hashes; compile/request result; output hash; syntax; validator/version; schema result; semantic result out of scope; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — frozen structured-output fixture requires an exact source, identity, and result receipt | UNTESTED — module 2 rule is reproducible but no production observation is claimed |
batch58-structured-output-m2-r6string-format | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=string-format; canonical/provider schema hashes; compile/request result; output hash; syntax; validator/version; schema result; semantic result out of scope; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — frozen structured-output fixture requires an exact source, identity, and result receipt | UNTESTED — module 2 rule is reproducible but no production observation is claimed |
First-party citation: Google Gemini API model documentation. Verified 2026-09-02; missing or conflicting joins fail closed.
Failure-and-repair receipt board
Deterministic formula / rule: repair = deterministic parse/validate/retry/hold rule; silent coercion never counts as native conformance.
Boundary: Owns protocol failure and repair receipts, not cost, quality, or extraction recommendations.
| Frozen scenario / field ID | Exact identity and evidence fields | Result | State |
|---|---|---|---|
batch58-structured-output-m3-r1request rejection | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=request rejection; failure class; provider signal; partial artifact; retry mutation; final syntax/schema state; cost/quality handoff; hold state; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — frozen structured-output fixture requires an exact source, identity, and result receipt | UNTESTED — module 3 rule is reproducible but no production observation is claimed |
batch58-structured-output-m3-r2refusal | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=refusal; failure class; provider signal; partial artifact; retry mutation; final syntax/schema state; cost/quality handoff; hold state; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — exact structured-output evidence join is not closed for “refusal” | FAIL CLOSED — manual, probe, or source evidence required |
batch58-structured-output-m3-r3length truncation | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=length truncation; failure class; provider signal; partial artifact; retry mutation; final syntax/schema state; cost/quality handoff; hold state; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — exact structured-output evidence join is not closed for “length truncation” | FAIL CLOSED — manual, probe, or source evidence required |
batch58-structured-output-m3-r4interrupted stream | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=interrupted stream; failure class; provider signal; partial artifact; retry mutation; final syntax/schema state; cost/quality handoff; hold state; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — exact structured-output evidence join is not closed for “interrupted stream” | FAIL CLOSED — manual, probe, or source evidence required |
batch58-structured-output-m3-r5syntactically valid schema failure | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=syntactically valid schema failure; failure class; provider signal; partial artifact; retry mutation; final syntax/schema state; cost/quality handoff; hold state; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — exact structured-output evidence join is not closed for “syntactically valid schema failure” | FAIL CLOSED — manual, probe, or source evidence required |
batch58-structured-output-m3-r6schema-valid semantic error | route=/compare/llm-structured-output; owner=structured-output; provider/seller/host/account/tier/endpoint/model/snapshot/region/feature/request=structured-output; scenario=schema-valid semantic error; failure class; provider signal; partial artifact; retry mutation; final syntax/schema state; cost/quality handoff; hold state; observed versus documented versus assumed inputs are labeled; artifact/schema/tool/job/result hashes and verification date=2026-09-02 | Unavailable — frozen structured-output fixture requires an exact source, identity, and result receipt | UNTESTED — module 3 rule is reproducible but no production observation is claimed |
First-party citation: Anthropic Messages API. Verified 2026-09-02; missing or conflicting joins fail closed.
Method and limitations: this board exposes deterministic rules, first-party citations, and dated evidence identities. It does not invent volume, coverage, entitlement, retention, residency, quota, capacity, feature support, quality, price, reliability, or legal conclusions. Run the structured-output evidence flow →