Reliable JSON from a large language model starts with more than the instruction “return JSON.” This structured output prompt guide explains how to define a usable schema, choose the right enforcement method, validate every response, and recover safely when an output is incomplete or malformed.
Overview
LLMs generate text, while software systems consume data structures. That difference is the source of many integration failures. A response can look correct to a person yet fail a parser because it contains an extra sentence, uses the wrong data type, omits a required property, or places an object where an array was expected.
The dependable approach is to treat structured output as a small data pipeline:
- Define the output contract with a JSON Schema or an equivalent specification.
- Describe the fields and allowed values in the prompt.
- Use provider-supported structured outputs when available.
- Parse and validate the response in application code.
- Retry, repair, or route the item for review when validation fails.
This is prompt engineering applied to an API boundary. The prompt influences meaning and behavior, but validation remains the final authority. Do not use a prompt as a substitute for parsing, schema validation, access control, or business-rule checks.
A practical JSON contract
Suppose an application extracts a support request from an email. A useful contract might require a category, urgency, summary, and list of requested actions:
{
"type": "object",
"additionalProperties": false,
"required": ["category", "urgency", "summary", "actions"],
"properties": {
"category": {
"type": "string",
"enum": ["billing", "technical", "account", "general"]
},
"urgency": {
"type": "string",
"enum": ["low", "normal", "high"]
},
"summary": {
"type": "string"
},
"actions": {
"type": "array",
"items": {"type": "string"}
}
}
}The schema makes several decisions explicit: which keys must exist, which values are allowed, whether unknown keys are rejected, and which fields are arrays. Those decisions are more useful than a vague instruction such as “summarize the email in JSON.”
Checklist by scenario
Scenario 1: You are extracting fields from text
- List every field the downstream system actually needs; remove fields added only for convenience.
- Choose a type for each field: string, number, boolean, array, or object.
- Define what happens when information is absent. Use
null, an empty array, or a documented default consistently. - Use enums for controlled categories rather than asking the model to invent labels.
- State whether values should be copied, normalized, or inferred.
- Require the model to distinguish missing information from an explicit negative statement.
For example, an extraction prompt can say: “Return only the JSON object matching the schema. Use null for a field that is not present in the source. Do not guess dates, identifiers, or prices. Preserve the source wording for the quoted text field.” This is a stronger system prompt example because it defines behavior around uncertainty, not just formatting.
Scenario 2: You are classifying content
- Provide the complete label set and define each label briefly.
- Require exactly one label when the task is single-class classification.
- Use an array only when multiple labels are genuinely valid.
- Add a confidence field only if the application has a defined use for it; a model-generated confidence value is not a calibrated probability by default.
- Include an
unknownorneeds_reviewoutcome when forcing a known label would be unsafe.
A compact classification response might look like this:
{
"label": "needs_review",
"reason": "The message mentions both billing and account access, and the requested action is unclear."
}Keep the reason for internal review rather than presenting it as proof that the classification is correct. If a human or another rule must verify the decision, make that workflow explicit.
Scenario 3: You need nested objects
- Define required properties at each nesting level.
- Keep nesting shallow unless the data model truly requires deeper structure.
- Specify whether an empty object or
nullrepresents “not available.” - Set limits for array length, string length, and numeric ranges in application validation.
- Test combinations of missing, empty, and contradictory values.
Nested schemas are especially prone to partial responses. A model may produce the parent object while omitting a child property. Validate recursively and report the exact path of the failure, such as customer.address.postal_code, so a recovery prompt can target the problem.
Scenario 4: You are connecting to a provider API
Use the strongest structured-output mechanism supported by the selected model and SDK, but verify its current restrictions before implementation. Provider features may differ in supported schema keywords, strictness, refusal behavior, streaming behavior, and token accounting.
| Method | Strength | Limitations | Best use |
|---|---|---|---|
| Native structured outputs | Can enforce or constrain a declared schema at the API layer. | Support and schema features vary by provider, model, and SDK. | Production workflows where parsing reliability matters. |
| JSON mode | Encourages valid JSON rather than free-form prose. | May not enforce required fields, enums, or the exact shape. | Simple responses with application-side validation. |
| Prompt-only formatting | Works across many interfaces and is easy to prototype. | Cannot guarantee syntax, completeness, or business rules. | Early experiments, human review, or fallback paths. |
Do not assume that a feature called “JSON mode” provides full JSON Schema enforcement. Read the provider’s current documentation, inspect the SDK behavior, and test the exact model configuration used by your application.
What to double-check
Prompt design
Place the output contract where the model can consistently use it. Identify the task, provide the input boundaries, define the schema, and state the output rule. A reusable template is:
You are a data extraction component.
Task:
Extract the requested fields from the input. Use only information supported by the input.
Output contract:
Return one JSON object matching the supplied schema. Return no markdown, commentary, or additional keys.
Use null when a required value is not present and the schema permits null.
Do not invent values. Preserve the requested units and date format.
Schema:
[insert schema or provider schema reference]
Input:
[insert delimited source text]For few-shot prompting examples, include only examples that demonstrate a difficult distinction, such as an absent value versus an explicit “no.” Keep examples consistent with the schema. An example that contains an extra key can teach the wrong contract.
Validation and recovery
Use a two-stage check. First, parse the response as JSON. Second, validate the resulting value against the schema and application rules. These checks answer different questions: parsing asks whether the syntax is valid; schema validation asks whether the structure is acceptable.
When validation fails, capture the original response, the validation path, the model identifier, the prompt version, and relevant request settings. A targeted retry is usually more useful than resending the same request:
The previous response failed validation.
Error path: actions[1]
Expected: a string
Received: an object
Return the corrected JSON object only. Preserve all valid fields and fix only the invalid value.Set a retry limit. If the response still fails, route it to a fallback model, deterministic parser, queue, or human review according to the risk of the task. Never silently coerce a malformed value when that could alter meaning.
Operational checks
- Keep prompt, schema, and parser versions together.
- Record validation failure rates separately from model latency and token usage.
- Test empty inputs, long inputs, multiple entities, conflicting statements, unusual Unicode, and truncated responses.
- Check how refusals, timeouts, tool calls, and streamed partial responses are represented.
- Apply size limits before sending untrusted or unexpectedly large input to the model.
- Use a JSON formatter or validator during development, but do not treat a formatter as a schema validator.
For a broader regression process, see the Prompt Testing and Evaluation Toolkit and the LLM Evaluation Guide.
Common mistakes
- Relying on “JSON only.” This may reduce commentary but does not guarantee the required shape. Pair the instruction with a schema and validation.
- Making every field required. A field that is not always available should be nullable or optional according to the application’s real data model.
- Allowing unrestricted categories. Free-form labels create downstream variations such as
high,urgent, andpriority_high. Use enums where the set is known. - Accepting extra keys without review. Additional fields can increase storage, confuse consumers, or hide prompt drift. Decide whether to reject or explicitly preserve them.
- Using model confidence as truth. A confidence number needs evaluation before it can support thresholds or automated decisions.
- Repairing JSON with string substitutions. Regex-based fixes can corrupt escaped characters, nested structures, and quoted content. Prefer a constrained retry or a real parser.
- Skipping regression tests after schema changes. Adding one field can affect token limits, examples, validation, and every downstream consumer.
If outputs vary unexpectedly, use a structured troubleshooting process rather than adding random prompt instructions. The guide to AI prompt debugging can help isolate whether the cause is the input, prompt, model, schema, or parser.
When to revisit
Revisit this checklist whenever the workflow or its inputs change. In particular, review the contract before seasonal planning cycles or other periods when volume, document formats, categories, or staffing assumptions may shift. A schema that worked for a small pilot may need limits, batching, or a review path when the input mix changes.
Also revisit structured output prompts when you:
- switch models, providers, SDK versions, or response settings;
- add, rename, remove, or reorder fields;
- change an enum, date format, unit, or null-handling rule;
- introduce retrieval, tools, memory, or agent steps;
- observe new validation failures, refusals, truncation, or latency issues;
- use the output for a higher-risk automated action.
Before release, run a fixed test set containing ordinary, ambiguous, incomplete, and adversarial inputs. Compare both validity and usefulness: a response can satisfy the schema while extracting the wrong value. Save representative failures and add them to regression tests. If the workflow includes retrieval, review the RAG tutorial; if it includes tools or agent loops, consult the guide on building AI agents with guardrails.
The practical rule is simple: define the contract, enforce it as close to the API as possible, validate independently, and make failure visible. That workflow will not make every LLM response correct, but it turns unpredictable text generation into a testable integration that can improve as the schema, prompts, and underlying tools evolve.