Structured Outputs
Overview
The use of structured outputs help you control how your LLMs response will be formatted. You can configure the output in Freeplay and test it in the prompt playground.
Why use Structured output?
Structred outputs provide consistency and safety to your AI outputs. This allows you to easily parse the output information in a consistent and expected manor. This can help eliminate parsing, reducing errors, add type saftey an make your integrations reliable and safe for use.
Supported Types
Freeplay supports two ways to get structred output from your LLMs:
OpenAI Structured Outputs
Guarantees schema compliance - The LLM will always match your exact JSON schema.
Best for:
- Production applications requiring reliable, consistent data structures
- Data feeding into typed systems or databases
- Extracting structured information from unstructured data
Supported models: OpenAI gpt-4o and later
Learn more about Structured Outputs →
JSON Mode
Ensures valid JSON - The LLM will output valid JSON without enforcing a specific structure.
Best for:
- Rapid prototyping and experimentation
- When you need JSON but don't require strict validation
- Working with models that don't support structured outputs
When using JSON mode, you must always instruct the model to produce JSON via some message in the conversation, for example via your system message. If you don't include an explicit instruction to generate JSON, the model may generate an unending stream of whitespace and the request may run continually until it reaches the token limit. To help ensure you don't forget, the API will throw an error if the string "JSON" does not appear somewhere in the context.
JSON mode will not guarantee the output matches any specific schema, only that it is valid and parses without errors. You should use Structured Outputs to ensure it matches your schema, or if that is not possible, you should use a validation library and potentially retries to ensure that the output matches your desired schema.
Comparison
| Feature | Structured Outputs | JSON Mode |
|---|---|---|
| Valid JSON output | ✓ Always | ✓ Always |
| Schema compliance | ✓ Guaranteed | ✗ Not guaranteed |
| Schema definition required | ✓ Yes (ex: Pydantic/Zod) | ✗ No |
| Flexibility | Lower (strict schema) | High |
| Validation needed | ✗ Automatic | ✓ Manual |
| Model support | OpenAI gpt-4o+ | Most OpenAI models |
| Supported providers | OpenAI, Azure OpenAI | OpenAI, Azure OpenAI |
| Best for | Production apps | Prototyping, flexible output |
| Error handling | Refusals programmatically detectable | Manual validation required |
Quick decision guide
Use Structured Outputs if:
- You're building a production application
- You need guaranteed field presence and types
- Data feeds directly into databases or typed systems
- You want to eliminate validation logic and retries
Use JSON Mode if:
- You're prototyping or experimenting
- You need flexibility over strict compliance
- You want JSON output without defining formal schemas
How Freeplay helps
Regardless of which approach you choose, Freeplay supports you by allowing to to quickly test and iterate against your output requirements in the prompt playground, render the structured outputs for easy review and analysis and Freeplay also allows you to record your structure defined in code and save it to your template for use.
Updated 8 days ago