Guides
Start typing to search…

Structured Outputs (OpenAI)

Structured outputs allow you to constrain LLM responses to follow a specific JSON schema. This ensures your application receives consistently formatted data that matches your expected structure, eliminating the need for complex parsing logic and reducing errors from malformed responses.

Key points

  • Define schemas in code: Use a json schema such as Pydantic (Python) or Zod (Node.js) to define your schemas as the source of truth
  • Pass to Freeplay on record: Include output_schema (in JSON format) in your RecordPayload when recording completions
  • Freeplay interprets automatically: Freeplay extracts and stores your schema for use in testing and observability
  • OpenAI format: OpenAI requires response_format with type json_schema, setting strict: true, and providing the schema
  • Keep code as source of truth: While you can view and edit schemas in the Freeplay UI, we recommend maintaining schemas in your codebase for consistency

How does Freeplay help with structured outputs?

Freeplay supports the complete lifecycle of working with structured outputs - by interpreting schemas passed via code and saving them with prompt templates to allowing you to view and manage output schemas alongside your prompt templates.

The SDK supports recording structured outputs and their associated schemas which are interpreted by Freeplay and can be associated with your prompt template. With the Freeplay web app, you can define output schemas alongside your prompt templates.

Managing your output schema with Freeplay

To get started from your code, simply have your prefedined schemas, see for example using Pydantic and Zod to define a very specific output:

from pydantic import BaseModel
from typing import List

class ResumeWorkExperience(BaseModel):
    company: str
    position: str
    start_date: str
    end_date: str
    description: str

class ResumeData(BaseModel):
    full_name: str
    email: str
    phone: str
    skills: List[str]
    work_experience: List[ResumeWorkExperience]
import { z } from "zod";

const ResumeWorkExperienceSchema = z.object({
    company: z.string(),
    position: z.string(),
    startDate: z.string(),
    endDate: z.string(),
    description: z.string(),
});

const ResumeDataSchema = z.object({
    fullName: z.string(),
    email: z.string(),
    phone: z.string(),
    skills: z.array(z.string()),
    workExperience: z.array(ResumeWorkExperienceSchema),
});

Pass schemas to Freeplay when recording

When you record an LLM interaction to Freeplay, include the output_schema parameter. Freeplay will automatically interpret and store your schema, making it available in the UI for testing and iteration.

from freeplay import Freeplay, RecordPayload, CallInfo, ResponseInfo

# After making your LLM call with structured outputs...
payload = RecordPayload(
    project_id=project_id,
    all_messages=all_messages,
    inputs=prompt_vars,
    session_info=session.session_info,
    prompt_version_info=formatted_prompt.prompt_info,
    call_info=CallInfo.from_prompt_info(formatted_prompt.prompt_info, start, end),
    response_info=ResponseInfo(is_complete=True),
    output_schema=ResumeData.model_json_schema()  # Pass your Pydantic schema
)

fpclient.recordings.create(payload)
import { z } from "zod";

// After making your LLM call with structured outputs...
await fpClient.recordings.create({
    projectId,
    allMessages: messages,
    sessionInfo: getSessionInfo(session),
    inputs: inputVariables,
    promptVersionInfo: formattedPrompt.promptInfo,
    callInfo: callInfo,
    responseInfo: { isComplete: true },
    outputSchema: z.toJSONSchema(ResumeDataSchema)  // Pass your Zod schema
});

That's it! Freeplay now has your structured output schema and you can leverage it throughout the platform.

Open a completion in the prompt editor

Once you have recorded a completion with your schema, you can have it automatically added to your Freeplay prompt template by opening the example in the prompt editor. This is when Freeplay will automatically interpret the structure and associate it with your prompt.

Now save this version of your prompt template and you are all done! You can now access the model parameter formatted_prompt.formatted_output_schema in your code to access the schema.

Benefits of this approach

By defining schemas in code and passing them to Freeplay, you get:

  • Source of truth in code: Your application code remains the authoritative definition of your schemas, ensuring consistency between development and production
  • Type safety: Leverage your language's type system and validation libraries
  • Easy testing: Run tests in Freeplay UI that respect your structured output format
  • Quick iteration: Test prompt variations in Freeplay while maintaining schema compliance
  • Observability: View structured outputs in the Freeplay dashboard with full schema context

Testing with structured outputs

Once your schema is recorded in Freeplay, you can run tests from the UI that will respect your structured output format. This makes it easy to:

  • Validate that prompt changes maintain schema compliance
  • Test against datasets to ensure consistent output structure
  • Debug schema validation issues before they reach production
  • Compare different prompt versions while enforcing the same output structure

When you run a test from the Freeplay UI on a prompt with an associated structured output schema, the test will automatically use that schema to validate responses.

Test run showing structured output validation

Viewing schemas in the Freeplay UI

You can view and inspect structured output schemas in the Freeplay UI, though we recommend keeping your code as the source of truth for schema definitions.

When viewing a completion or prompt template in Freeplay, you'll see the associated output schema displayed. This is helpful for:

  • Understanding what structure was used for a particular completion
  • Reviewing schema definitions when debugging issues
  • Sharing schema information with team members

While it's possible to define or modify schemas directly in the Freeplay UI, we recommend using this primarily for exploration and testing. For production use, maintain your schemas in code to ensure consistency and version control.

Output schema view in Freeplay UI

Complete code examples

See a complete end to end example here.

Important notes

  • OpenAI only: At this time, structured outputs are only supported with OpenAI models. Support for additional providers is coming soon.
  • Schema normalization: Freeplay automatically adds "additionalProperties": false to all object types in your schema. This ensures the LLM doesn't add unexpected fields to your structured output.
  • Validation: When you add an output schema to a prompt template, Freeplay validates that your selected model supports structured outputs. You'll receive an error if you try to use structured outputs with an unsupported model.
  • Strict mode: OpenAI's structured outputs require strict: true in the json_schema configuration. This enforces exact schema compliance.
  • Nested schemas: You can use nested objects and arrays in your schemas for complex structured outputs.
  • Optional vs required fields: Use the required array to specify which fields must be present. Fields not in the required array are optional.
  • Programmatic schemas: You can define schemas either in your prompt templates or programmatically using JSON schemas such as Pydantic (Python) or Zod (Node.js). Both approaches work with Freeplay's recording and observability.

Updated about 18 hours ago