Structured Outputs
Control how your LLM outputs JSON data with Freeplay. Choose between strict schema enforcement or flexible JSON output based on your needs.
Why use JSON output?
JSON is one of the most widely used formats for applications to exchange data. Using JSON output with LLMs helps you:
- Eliminate parsing complexity: Get data in a format your application can directly consume
- Reduce errors: Avoid string parsing and malformed response handling
- Enable type safety: Use structured data with your application's type system
- Simplify integration: Feed LLM outputs directly into databases, APIs, or UI components
Choose your approach
Freeplay supports two ways to get JSON output from your LLMs:
OpenAI Structured Outputs (Recommended)
Guarantees schema compliance - The LLM will always match your exact JSON schema.
Best for:
- Production applications requiring reliable, consistent data structures
- Data feeding into typed systems or databases
- Extracting structured information from unstructured data
Supported models: OpenAI gpt-4o and later
Learn more about Structured Outputs →
JSON Mode
Ensures valid JSON - The LLM will output valid JSON without enforcing a specific structure.
Best for:
- Rapid prototyping and experimentation
- When you need JSON but don't require strict validation
- Working with models that don't support structured outputs
When using JSON mode, you must always instruct the model to produce JSON via some message in the conversation, for example via your system message. If you don't include an explicit instruction to generate JSON, the model may generate an unending stream of whitespace and the request may run continually until it reaches the token limit. To help ensure you don't forget, the API will throw an error if the string "JSON" does not appear somewhere in the context.
JSON mode will not guarantee the output matches any specific schema, only that it is valid and parses without errors. You should use Structured Outputs to ensure it matches your schema, or if that is not possible, you should use a validation library and potentially retries to ensure that the output matches your desired schema.
Comparison
| Feature | Structured Outputs | JSON Mode |
|---|---|---|
| Valid JSON output | ✓ Always | ✓ Always |
| Schema compliance | ✓ Guaranteed | ✗ Not guaranteed |
| Schema definition required | ✓ Yes (ex: Pydantic/Zod) | ✗ No |
| Flexibility | Lower (strict schema) | High |
| Validation needed | ✗ Automatic | ✓ Manual |
| Model support | OpenAI gpt-4o+ | Most OpenAI models |
| Supported providers | OpenAI, Azure OpenAI | OpenAI, Azure OpenAI |
| Best for | Production apps | Prototyping, flexible output |
| Error handling | Refusals programmatically detectable | Manual validation required |
Quick decision guide
Use Structured Outputs if:
- You're building a production application
- You need guaranteed field presence and types
- Data feeds directly into databases or typed systems
- You want to eliminate validation logic and retries
Use JSON Mode if:
- You're prototyping or experimenting
- You need flexibility over strict compliance
- You want JSON output without defining formal schemas
How Freeplay helps
Regardless of which approach you choose, Freeplay provides:
- Unified recording: Record both structured outputs and JSON mode completions
- Schema visibility: View output schemas alongside completions in observability
- Testing support: Run tests that respect your output format
- Prompt management: Configure output formats in prompt templates
- Observability: Monitor JSON output quality and schema compliance over time
Updated about 11 hours ago