API Reference

This guide provides comprehensive documentation for the Freeplay HTTP API, including authentication, available endpoints, request/response formats, and code examples.

Authentication

Your API root is the URL from your Freeplay instance plus the suffix /api/v2/. For example: https://app.freeplay.ai/api/v2

Custom Domain

If you are a private deployment, your API root may be something like https://acme.freeplay.ai/api/v2.

API Keys

Freeplay authenticates your API requests using your API Key, which can be managed through the Freeplay application at https://app.freeplay.ai/settings/api-access. To authenticate, provide an API key in the Authorization header:

Authorization: Bearer {freeplay_api_key}

API Errors

When using the Freeplay HTTP API, you may encounter various error responses. We use standard HTTP error codes: 200 for success, 400 for bad requests, 401 for authorization errors, and 500 for server errors. Most errors include an HTTP body in JSON format describing what went wrong and how to fix the issue.

Do not retry errors with codes in the 400-499 range, as they are typically client errors that will not be fixed by retrying.

For 500 errors, retry the request up to three times with at least a 5-second delay between attempts. These are often transient issues that may resolve on retry.

Common Error Codes

400 Bad Request

The request was malformed or contained invalid data. This occurs when the request is not proper JSON or contains invalid field values.

{
  "message": "Session ID 456 is not a valid uuid4 format."
}

401 Unauthorized

The API request does not contain valid bearer authentication with an API key.

{
  "message": "Requests must be authenticated with a valid API key."
}

404 Not Found

One of the IDs passed in the request is invalid or the object does not exist. This can happen if the referenced object was deleted or if the caller does not have access to the object.

{
  "message": "Project not found"
}

500 Server Error

An error occurred in the Freeplay server. Our team is automatically notified. These are often transient issues that can be resolved by retrying the request after a delay.

{
  "message": "An unexpected error occurred. Please try again later."
}

When encountering errors, verify your request parameters and payload match the API specifications. If issues persist, contact Freeplay support for assistance.

Projects

A project in Freeplay represents a meaningful separation between your AI products. Most commonly, teams organize projects by AI product for easier tracking.

List All Projects

GET /projects/all Retrieves all projects in your Freeplay workspace.

This endpoint does not work with project-scoped API keys - Private projects are not returned

curl --location 'https://app.freeplay.ai/api/v2/projects/all' \
  --header 'Authorization: Bearer <APIKEY>'

Sample Response

{
  "projects": [
    {
      "id": "<id>",
      "name": "FreeplayChat"
    },
    {
      "id": "<id>",
      "name": "Freeplay Support Agent"
    },
    {
      "id": "<id>",
      "name": "Getting Started Project"
    }
  ]
}

Prompt Templates

A prompt template in the Freeplay platform is a pre-defined structure for creating prompts that are used to interact with LLMs. Base URL: /api/v2/projects/<project-id>/prompt-templates

Retrieve a Prompt Template by Name

POST /name/<template-name> You can retrieve a prompt template in one of three forms:

Raw: Returns the prompt template without input variable values inserted
Bound: Returns the prompt template with input variable values inserted in a consistent format. Pass your input variables in the request body.
Formatted (most common): Returns the prompt template with input variable values inserted and content formatted according to your LLM provider. Pass format=true as a query parameter and pass your input variables in the request body.

Query Parameters

Parameter	Type	Description	Required
environment	`str`	The environment tag of the prompt you want to fetch	No (Default: `latest`)
format	`boolean`	Indicates whether to return a Formatted or Bound Prompt	No

Retrieve a Formatted Prompt

curl 'https://app.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/prompt-templates/name/album_bot?environment=prod&format=true' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer TOKEN' \
  --data '{
    "pop_star": "justin beiber"
  }'

Sample Response

{
  "format_version": 2,
  "formatted_content": [
    {
      "content": "You are a helpful bot that generates album names\nGenerate a two word album name in the style of justin beiber",
      "role": "user"
    }
  ],
  "formatted_tool_schema": [
    {
      "function": {
        "description": "Get weather of a location",
        "name": "weather_of_location",
        "parameters": {
          "properties": {
            "location": {
              "description": "Location to get the weather for",
              "type": "string"
            }
          },
          "required": ["location"],
          "type": "object"
        }
      },
      "type": "function"
    }
  ],
  "metadata": {
    "flavor": "openai_chat",
    "model": "gpt-3.5-turbo-0125",
    "params": {
      "max_tokens": 100,
      "temperature": 0.2
    },
    "provider": "openai",
    "provider_info": {}
  },
  "prompt_template_id": "5cccc8e6-b163-4094-8bd2-90030f151ec8",
  "prompt_template_name": "album_bot",
  "prompt_template_version_id": "f503c15e-2f0f-4ce4-b443-4c87d0b6435d",
  "system_content": null
}

Retrieve a Bound Prompt

curl 'https://app.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/prompt-templates/name/album_bot?environment=prod' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer TOKEN' \
  --data '{
    "pop_star": "justin beiber"
  }'

Sample Response

{
  "bound_content": [
    {
      "content": "You are a helpful bot that generates album names\nGenerate a two word album name in the style of justin beiber",
      "role": "user"
    }
  ],
  "format_version": 2,
  "metadata": {
    "flavor": "openai_chat",
    "model": "gpt-3.5-turbo-0125",
    "params": {
      "max_tokens": 100,
      "temperature": 0.2
    },
    "provider": "openai",
    "provider_info": {}
  },
  "prompt_template_id": "5cccc8e6-b163-4094-8bd2-90030f151ec8",
  "prompt_template_name": "album_bot",
  "prompt_template_version_id": "f503c15e-2f0f-4ce4-b443-4c87d0b6435d",
  "tool_schema": [
    {
      "description": "Get weather of a location",
      "name": "weather_of_location",
      "parameters": {
        "additionalProperties": false,
        "properties": {
          "location": {
            "description": "Location to get the weather for",
            "type": "string"
          }
        },
        "required": ["location"],
        "type": "object"
      }
    }
  ]
}

Retrieve a Raw Prompt Template

curl 'https://app.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/prompt-templates/name/album_bot?environment=prod' \
  --header 'Authorization: Bearer TOKEN'

Sample Response

{
  "content": [
    {
      "content": "You are a helpful bot that generates album names\nGenerate a two word album name in the style of {{pop_star}}",
      "role": "user"
    }
  ],
  "format_version": 2,
  "metadata": {
    "flavor": "openai_chat",
    "model": "gpt-3.5-turbo-0125",
    "params": {
      "max_tokens": 100,
      "temperature": 0.2
    },
    "provider": "openai",
    "provider_info": {}
  },
  "prompt_template_id": "5cccc8e6-b163-4094-8bd2-90030f151ec8",
  "prompt_template_name": "album_bot",
  "prompt_template_version_id": "f503c15e-2f0f-4ce4-b443-4c87d0b6435d",
  "tool_schema": [
    {
      "description": "Get weather of a location",
      "name": "weather_of_location",
      "parameters": {
        "additionalProperties": false,
        "properties": {
          "location": {
            "description": "Location to get the weather for",
            "type": "string"
          }
        },
        "required": ["location"],
        "type": "object"
      }
    }
  ]
}

Retrieve All Prompt Templates in an Environment

GET /all/<environment-name>

curl 'https://app.freeplay.ai/api/v2/projects/<project-id>/prompt-templates/all/latest' \
  --header 'Authorization: Bearer TOKEN'

Sample Response

{
  "prompt_templates": [
    {
      "content": [
        {
          "content": "You are a helpful bot that generates album names\nRespond with only the album name",
          "role": "system"
        },
        {
          "content": "Generate a two word album name in the style of {{pop_star}}",
          "role": "user"
        }
      ],
      "format_version": 2,
      "metadata": {
        "flavor": "anthropic_chat",
        "model": "claude-3-opus-20240229",
        "params": {
          "max_tokens": 256,
          "temperature": 0.5
        },
        "provider": "anthropic",
        "provider_info": {}
      },
      "prompt_template_id": "5cccc8e6-b163-4094-8bd2-90030f151ec8",
      "prompt_template_name": "album_bot",
      "prompt_template_version_id": "eb934c9f-ee1e-42be-a9cb-d193f5d3aa54"
    },
    {
      "content": [
        {
          "content": "You are a news article summarization agent. Your job is to find the big idea in an article, distill it into compelling content & write a catchy summary that people want to click. You will be provided with the article title, source url and the article text\nThe scraped source content of the URL for the article is provided. ",
          "role": "system"
        },
        {
          "content": "Here is the scraped content from the web page:\n--------\n{{article_title}}\n{{article_text}}\n{{source_url}}\n--------\n\nSome additional instructions follow. Please follow these carefully.\n\nThere may be user comments at the bottom or other superfluous information included from the scraped content. Ignore those and focus on the body of the article. \n\nAt the end of each response, after a newline always include the original URL for the article so people can read the article. \n\nWhen sharing the URL, include it after two line breaks and use this format starting with \"Read more\" and stripping the http:// at the start of the url: Read more: www.example.com\nGiven this article, respond with a Tweet of no more than 200 characters. Never include hashtags. A relevant emoji is ok. Do not include quotes around any content.\n\nAfter a newline, include the original URL for the article so people can read the article.",
          "role": "user"
        }
      ],
      "format_version": 2,
      "metadata": {
        "flavor": "openai_chat",
        "model": "gpt-3.5-turbo-16k-0613",
        "params": {
          "max_tokens": 2000,
          "temperature": 0.5
        },
        "provider": "openai",
        "provider_info": {}
      },
      "prompt_template_id": "9daa7233-16d3-48b3-a739-ddba66742b08",
      "prompt_template_name": "article_summarizer",
      "prompt_template_version_id": "9f9f9bfe-ed53-449d-a13c-5f88198c0f8e"
    }
  ]
}

Retrieve a Prompt Template by Version ID

POST /id/<template-id>/versions/<template-version-id> Retrieves a prompt template using a specific version ID. This is useful when you want to pin your code to a specific prompt version regardless of actions taken in the Freeplay app. The same Raw, Bound, and Formatted options exist when retrieving by version ID as they do when retrieving by name.

curl 'https://app.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/prompt-templates/id/0053c449-c736-4557-a3d4-bf3b852d0604/versions/42d301b2-cdc3-48d5-829b-b892561017ff' \
  --header 'Authorization: Bearer TOKEN'

Sample Response

{
  "content": [
    {
      "content": "You are a technical customer support agent for a product called freeplay. Your job is to help users answer questions about freeplay.\nYou will often be provided with information that will help you answer the question. If the information is relevant then use it to answer the question and end your response with \"See <source> for more details\" where <source> is a valid HTML formatted hyperlink to the source URL.\n\nIf the information provided is not relevant then ignore it.\n\nUser Query: {{question}}\n----\nSupporting Information: {{supporting_information}}",
      "role": "user"
    }
  ],
  "format_version": 2,
  "metadata": {
    "flavor": "openai_chat",
    "model": "gpt-3.5-turbo-16k-0613",
    "params": {
      "max_tokens": 2000,
      "temperature": 0.5
    },
    "provider": "openai",
    "provider_info": {}
  },
  "prompt_template_id": "0053c449-c736-4557-a3d4-bf3b852d0604",
  "prompt_template_name": "rag-qa",
  "prompt_template_version_id": "42d301b2-cdc3-48d5-829b-b892561017ff"
}

Create a Prompt Template Version

POST /id/<template-id>/versions Adds a new version to an existing prompt template. The new version is immediately deployed to the latest environment unless you specify different environments. Parameters

Parameter	Type	Description	Required
`content`	`str`	JSON-encoded array of chat-style messages, each with `role` and `content`. Variables use `{{variable}}` syntax.	Yes
`model`	`str`	Model name (e.g. `gpt-4o-2024-05-13`)	Yes
`provider`	`str`	Provider key (`openai`, `anthropic`, `vertex`, etc.)	Yes
`llm_parameters`	`dict[str, number, bool]`	Overrides for model parameters (e.g. `temperature`, `max_tokens`)	Yes
`tool_schema`	`list[NormalizedToolSchema]`	JSON schema(s) for function/tool calls. Each name must be unique.	No
`version_name`	`str`	Human-readable name shown in the UI	No
`version_description`	`str`	Longer description shown in the UI	No
`environments`	`list[str]`	Deploy this version to the specified environments (defaults to `["latest"]`)	No

Successful calls return 201 Created with the new version’s metadata.

Example Request

curl -X POST \
  'https://app.freeplay.ai/api/v2/projects/9cf924e8-4474-442a-bf9f-53f8415a4bd2/prompt-templates/id/d500b46e-0ae2-46c0-ac0e-0c18feb180fb/versions' \
  --header 'Authorization: Bearer TOKEN' \
  --header 'Content-Type: application/json' \
  --data '{
    "content": "[{\"role\": \"system\", \"content\": \"You are a helpful assistant.\"}, {\"role\": \"user\", \"content\": \"Hello {{name}}!\"}]",
    "provider": "openai",
    "model": "gpt-4o-2024-05-13",
    "llm_parameters": {
      "temperature": 0.2,
      "max_tokens": 256
    },
    "tool_schema": [
      {
        "name": "get_weather",
        "description": "Get weather information",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string"
            }
          },
          "required": ["location"]
        }
      }
    ],
    "version_name": "v2 – temp 0.2",
    "version_description": "Updated version with lower temperature",
    "tags": ["latest"]
  }'

Sample Response (201)

{
  "content": [
    {
      "content": "You are a helpful assistant.",
      "role": "system"
    },
    {
      "content": "Hello {{name}}!",
      "role": "user"
    }
  ],
  "created_at": 1754585113,
  "format_version": 3,
  "metadata": {
    "flavor": "openai_chat",
    "model": "gpt-4o-2024-05-13",
    "params": {
      "max_tokens": 256,
      "temperature": 0.2
    },
    "provider": "openai",
    "provider_info": {}
  },
  "project_id": "9cf924e8-4474-442a-bf9f-53f8415a4bd2",
  "prompt_template_id": "d500b46e-0ae2-46c0-ac0e-0c18feb180fb",
  "prompt_template_name": "testing234",
  "prompt_template_version_id": "3a583bb3-901e-4f78-8364-f23b2f260019",
  "tool_schema": [
    {
      "description": "Get weather information",
      "name": "get_weather",
      "parameters": {
        "properties": {
          "location": {
            "type": "string"
          }
        },
        "required": ["location"],
        "type": "object"
      }
    }
  ],
  "version_description": "Updated version with lower temperature",
  "version_name": "v2 – temp 0.2"
}

Validation & Errors

400 Bad Request

"Invalid prompt template content format …" – content is not valid JSON or message list
"No matching model found …" – unknown model/provider
"This provider does not support tool calls …" – model lacks tool-use support
"Duplicate tool name found …" – duplicate names in tool_schema

404 Not Found

template_id does not exist or you lack access

Update Environments for a Prompt Template Version

POST /id/<template-id>/versions/<version-id>/environments Assigns an existing prompt template version to one or more environments.

Environments must already exist in your account - The latest environment is managed by the system and is ignored if provided - This operation is idempotent and additive: it ensures the version is tagged with the provided environments but does not remove any previously assigned environments

Parameters

Parameter	Type	Description	Required
`environments`	`list[string]`	Environment names to assign (e.g., `["staging", "prod"]`). `latest` is ignored.	Yes

curl -X POST \
  'https://app.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/prompt-templates/id/0053c449-c736-4557-a3d4-bf3b852d0604/versions/42d301b2-cdc3-48d5-829b-b892561017ff/environments' \
  --header 'Authorization: Bearer TOKEN' \
  --header 'Content-Type: application/json' \
  --data '{"environments": ["staging", "prod", "latest"]}'

Sample Response (200)

{}

Validation & Errors

400 Bad Request

"Environment '<name>' not found …" – the environment does not exist for your account

404 Not Found

The template_id or version_id does not exist, or you lack access

Agents

An agent in Freeplay is a process in your application that makes use of multiple LLM calls to generate a single response. Base URL: /api/v2/projects/<project-id>/agents

List Agents

GET / Returns agents in descending chronological order (newest first). Use pagination to navigate beyond the initial page. Parameters

Parameter	Type	Description	Required
page	`int`	The page number of agents to retrieve	No (Default: `1`)
page_size	`int`	The number of agents to include in the response (maximum: 100)	No (Default: `30`)
name	`string`	Filter agents by exact name match (case-sensitive)	No

curl --location 'https://app.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/agents' \
  --header 'Authorization: Bearer TOKEN'

Sample Response

{
  "data": [
    {
      "id": "deadbeef-dead-0000-beef-deadbeefdead",
      "name": "customer_support_agent"
    },
    {
      "id": "f00dbabe-f00d-1111-babe-f00dbabef00d",
      "name": "sales_assistant"
    }
  ],
  "pagination": {
    "page": 1,
    "page_size": 30,
    "has_next": false
  }
}

Example with Filters

# Get agents with a specific name
curl --location 'https://app.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/agents?name=customer_support_agent' \
  --header 'Authorization: Bearer TOKEN'

# Get second page of agents with custom page size
curl --location 'https://app.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/agents?page=2&page_size=50' \
  --header 'Authorization: Bearer TOKEN'

Sessions

Sessions are a collection of one or more LLM completions, enabling you to tie completions together in whatever logical grouping makes sense for your application (multi-turn chat, chains, agent flows, etc.). Base URL: /api/v2/projects/<project-id>/sessions

Record a Completion

POST /<session-id>/completions Records a completion to a session. This endpoint allows you to send LLM interaction data to Freeplay for observability and analysis. Request Payload

Parameter	Type	Description	Required
messages	`list[{"role": Literal["user", "assistant"], "content": str}]`	List of messages sent to the LLM. Note for Anthropic: messages must exclude message fields other than `role` and `content`.	Yes
tool_schema	`list[{name: str, description: str, parameters: dict[str, any]}]`	Tool schema used with the completion	No
inputs	`dict[str, any]`	Input variables used to build the formatted prompt. Values must not be null.	Yes
session_info	`{custom_metadata: dict[str, str]}`	Information to be associated with the session	No
prompt_info	`{prompt_template_version_id: UUID, environment: str}`	Information associated with the prompt template	Yes
call_info	`{start_time: float, end_time: float, model: str, provider: str, provider_info: dict[str, any], llm_parameters: dict[str, any], usage: {prompt_tokens: int, completion_tokens: int}, api_style: 'batch' or 'default'}`	Information associated with the LLM call. Note: If `usage` is included, the token count will be used directly to calculate costs. If empty, Freeplay will estimate token counts using Tiktoken.	No
test_run_info	`{test_run_id: UUID, test_case_id: UUID}`	Information for the associated test run	No
completion_id	`UUID`	The UUIDv4 that should be used as the ID for the completion. Valid for sending completion feedback.	No
eval_results	`dict[str, bool or float]`	Code evaluations for a completion. These will be displayed alongside model-graded evals and human labels for a completion.	No

curl 'https://app.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/sessions/f503c15e-2f0f-4ce4-b443-4c87d0b6435d/completions' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer TOKEN' \
  --data '{
    "messages": [
      {
        "content": "You are a helpful bot that generates album names\nGenerate a two word album name in the style of Taylor Swift",
        "role": "user"
      },
      {
        "role": "assistant",
        "content": "\"Rainy Melodies\""
      }
    ],
    "inputs": {
      "pop_star": "Taylor Swift"
    },
    "prompt_info": {
      "prompt_template_version_id": "f503c15e-2f0f-4ce4-b443-4c87d0b6435d",
      "environment": "prod"
    },
    "eval_results": {
      "includes_required_keyword": true,
      "toxicity_score": 0.28
    }
  }'

Sample Response

{
  "completion_id": "707bc301-85e9-4f02-aa97-faba8cd7774a"
}

Record an LLM Interaction End to End

import requests
from dotenv import load_dotenv
import os
import json
import uuid

load_dotenv("../.env")
project_id = os.getenv("FREEPLAY_PROJECT_ID")
freeplay_api_root = f"https://app.freeplay.ai/api/v2/projects/{project_id}"
openai_api_url = "https://api.openai.com/v1/chat/completions"

## PROMPT ##
# fetch the prompt
prompt_name = "album_bot"
input_variables = {"pop_star": "Taylor Swift"}
prompt_req = requests.post(
    url=f"{freeplay_api_root}/prompt-templates/name/{prompt_name}",
    headers={
        "Authorization" : f"Bearer {os.getenv('FREEPLAY_KEY')}"
    },
    params={
        "environment": "prod",
        "format": "true"
    },
    data=json.dumps(input_variables)
)
formatted_prompt = prompt_req.json()

## LLM ##
completion_payload = {
        "model": formatted_prompt["metadata"]["model"],
        "messages": formatted_prompt["formatted_content"],
        **formatted_prompt["metadata"]["params"]
}

completion_req = requests.post(
    url=openai_api_url,
    headers={
        "Authorization" : f"Bearer {os.getenv('OPENAI_API_KEY')}",
        "Content-Type": "application/json"
    },
    data=json.dumps(completion_payload)
)
response_message = completion_req.json()['choices'][0]['message']

## RECORD ##
# record the interaction to freeplay
# add the LLM response to the message queue
messages = formatted_prompt["formatted_content"]
messages.append(response_message)

# construct the payload
record_payload = {
    "messages": messages,
    "inputs": input_variables,
    "prompt_info": {
        "prompt_template_version_id": formatted_prompt["prompt_template_version_id"],
        "environment": "prod"
    }
}

# create a session id
session_id = str(uuid.uuid4())

# record to freeplay
requests.post(
    url=f"{freeplay_api_root}/sessions/{session_id}/completions",
    headers={
        "Authorization": f"Bearer {os.getenv('FREEPLAY_KEY')}",
        "Content-Type": "application/json"
    },
    data = json.dumps(record_payload)
)

Retrieve Existing Sessions/Completions

GET / Retrieves sessions, including their metadata and completions, starting from most recent first. Parameters

Parameter	Type	Description	Required
page	`int`	The page number of sessions to retrieve	No (Default: `1`)
page_size	`int`	The number of sessions to include in the request (maximum: 100)	No (Default: `10`)
from_date	`str`	Earliest date for which sessions will be returned (inclusive). Format: `YYYY-MM-DD`	No
to_date	`str`	Latest date for which sessions will be returned (exclusive). Format: `YYYY-MM-DD`	No
test_list	`str`	Filters to sessions in the given dataset. Pass in the name of the dataset.	No
test_run_id	`str`	Filters to sessions in the given test run	No
prompt_name	`str`	Filters to sessions associated with the given prompt name	No
review_queue_id	`str`	Filters to sessions in the given review queue	No
custom_metadata.*	`dict`	Filters to sessions that have the associated metadata value at the session level	No

curl --header "Authorization: Bearer $FREEPLAY_API_KEY" \
  "https://app.freeplay.ai/api/v2/projects/$FREEPLAY_PROJECT_ID/sessions"

Sample Response

[
  {
    "custom_metadata": {},
    "messages": [
      {
        "client_evaluation_results": [],
        "completion_id": "0202ae57-1098-4d3e-94f7-1fbb034fbed5",
        "customer_feedback": {},
        "environment": "latest",
        "evaluation_results": [],
        "format_version": 3,
        "input_variables": {
          "question": "How do I bundle prompts so my system is reliable?"
        },
        "message_type": "completion",
        "model_name": "claude-2.1",
        "prompt": [
          {
            "content": "You are a technical customer support agent for a product called freeplay.",
            "role": "system"
          },
          {
            "content": "How do I bundle prompts so my system is reliable?",
            "role": "user"
          }
        ],
        "prompt_template_name": "my-prompt",
        "provider_name": "anthropic",
        "response": "You bundle prompts by downloading them with the python SDK, packaging them with your application, and using the FilesystemTemplateResolver when using the Freeplay SDK.",
        "trace_id": null
      }
    ],
    "session_id": "85d7d393-4e85-4664-8598-5dd91dc75b5b",
    "start_time": "2024-07-05T14:33:02.721000"
  }
]

Additional Filtering Examples

The following example demonstrates using multiple filters:

from_date: 2025-08-01
to_date: 2025-08-22
page: 1
page_size: 70 (default is 10, max is 100)
custom_metadata.is_llm_only: false

curl --header "Authorization: Bearer $FREEPLAY_API_KEY" \
  --location 'https://app.freeplay.ai/api/v2/projects/$FREEPLAY_PROJECT_ID/sessions?from_date=2025-08-01&to_date=2025-08-22&page=1&page_size=70&custom_metadata.is_llm_only=false'

Delete a Session

DELETE /api/v2/projects/<project-id>/sessions/<session-id> Deletes a session and all associated completions.

Deleting a session is permanent and cannot be undone.

curl --request DELETE \
  'https://app.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/sessions/a903c15e-2f0f-4ce4-b743-4c87d0c64918' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer TOKEN'

Test Runs

Test runs in Freeplay provide a structured way to generate batch tests for your language models using saved datasets managed on the Freeplay server. Base URL: /api/v2/projects/<project-id>/test-runs

Create a Test Run

POST / Creates a new test run from an existing dataset. Request Payload

Parameter	Type	Description	Required
dataset_name	`str`	The name of the dataset for which you want to create a test run	Yes
include_outputs	`boolean`	Determines if outputs are returned for test cases	No (Default: `true`)
test_run_name	`str`	Name for the test run, to be displayed in the Freeplay app	No
test_run_description	`str`	Description for the test run, to be displayed in the Freeplay app	No

curl 'https://app.freeplay.ai/api/v2/projects/<project-id>/test-runs' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer TOKEN' \
  --data '{
    "dataset_name": "Example Tests"
  }'

Sample Response

{
  "test_cases": [
    {
      "output": null,
      "test_case_id": "91e60c9e-fbaa-4990-b4cc-7a8bd067f298",
      "variables": {
        "question": "How do test runs relate to test cases?",
        "supporting_information": "([SearchChunk(title='Test Runs', link='https://docs.freeplay.ai/docs/test-runs-with-freeplay', content='Test Runs')..."
      }
    },
    {
      "output": null,
      "test_case_id": "c45dbc01-a177-449b-ab66-bb5a469748fe",
      "variables": {
        "question": "Write me some typescript code to run a new Test Run in freeplay",
        "supporting_information": "[{\"id\":88,\"version\":5,\"score\":0.8668302,\"payload\":{\"description\":\"Ship better products with LLMs...."
      }
    }
  ],
  "test_run_description": "",
  "test_run_id": "bd3eb06c-f93b-46a4-aa3b-d240789c8a06",
  "test_run_name": ""
}

Execute a Test Run end to end

import requests
from dotenv import load_dotenv
import os
import json
import uuid

load_dotenv("../.env")
project_id = "45e20940-322a-4cb8-810f-3934a56aa5e2"
freeplay_api_root = f"https://app.freeplay.ai/api/v2/projects/{project_id}"
openai_api_url = "https://api.openai.com/v1/chat/completions"

# create the test run
test_run_req = requests.post(
    url= freeplay_api_root + "/test-runs",
    headers={
        "Authorization": f"Bearer {os.getenv('FREEPLAY_KEY')}",
        "Content-Type": "application/json"
    },
    data=json.dumps({
        "dataset_name": "Example Tests"
    })
)
test_run = test_run_req.json()

prompt_name = "rag-qa"
# loop over each test case in the dataset
for test_case in test_run["test_cases"]:
    # fetch the formatted prompt
    prompt_req = requests.post(
        url=f"{freeplay_api_root}/prompt-templates/name/{prompt_name}",
        headers={
            "Authorization" : f"Bearer {os.getenv('FREEPLAY_KEY')}"
        },
        params={
            "environment": "prod",
            "format": "true"
        },
        data=json.dumps(test_case['variables'])
    )
    formatted_prompt = prompt_req.json()

    # execute the completion
    completion_payload = {
        "model": formatted_prompt["metadata"]["model"],
        "messages": formatted_prompt["formatted_content"],
        **formatted_prompt["metadata"]["params"]
    }
    completion_req = requests.post(
        url=openai_api_url,
        headers={
            "Authorization" : f"Bearer {os.getenv('OPENAI_API_KEY')}",
            "Content-Type": "application/json"
        },
        data=json.dumps(completion_payload)
    )
    response_message = completion_req.json()['choices'][0]['message']

    # record to freeplay
    messages = formatted_prompt["formatted_content"]
    messages.append(response_message)

    record_payload = {
        "messages": messages,
        "inputs": test_case['variables'],
        "prompt_info": {
            "prompt_template_version_id": formatted_prompt["prompt_template_version_id"],
            "environment": "prod"
        },
        # include the test run info to link the session to the test run
        "test_run_info": {
            "test_run_id": test_run["test_run_id"],
            "test_case_id": test_case["test_case_id"]
        }
    }

    # create a session
    session_id = str(uuid.uuid4())
    record_res = requests.post(
        url=f"{freeplay_api_root}/sessions/{session_id}/completions",
        headers={
            "Authorization": f"Bearer {os.getenv('FREEPLAY_KEY')}",
            "Content-Type": "application/json"
        },
        data=json.dumps(record_payload)
    )

Retrieve Test Run Results

GET /id/<test-run-id>

curl 'https://app.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/test-runs/id/2a9dd8bd-6c29-47c8-9ca4-427f73174881' \
  --header 'Authorization: Bearer TOKEN'

Sample Response

{
  "created_at": 1733770879,
  "description": "",
  "id": "2a9dd8bd-6c29-47c8-9ca4-427f73174881",
  "model_name": "gpt-4o-mini-2024-07-18",
  "name": "132--after-concurrency",
  "prompt_name": "rag-qa",
  "prompt_version": "gpt-4o-mini",
  "sessions_count": 132,
  "summary_statistics": {
    "auto_evaluation": {
      "Answer Accuracy": {
        "1": 104,
        "2": 9,
        "4": 9,
        "5": 10
      },
      "Answer Faithfulness": {
        "no": 126,
        "yes": 6
      },
      "Context Relevance": {
        "1": 116,
        "4": 2,
        "5": 14
      }
    },
    "client_evaluation": {},
    "human_evaluation": {}
  }
}

List Test Runs

GET / Returns test runs in descending chronological order. Use pagination to navigate beyond the most recent 100. Parameters

Parameter	Type	Description	Required
page	`int`	The page number of test runs to retrieve (maximum page size: 100)	No (Default: `1`)
page_size	`int`	The number of test runs to include in the request	No (Default: `100`)

curl --location 'https://app.freeplay.ai/api/v2/projects/45e20940-322a-4cb8-810f-3934a56aa5e2/test-runs' \
  --header 'Authorization: Bearer TOKEN'

Sample Response

{
  "test_runs": [
    {
      "created_at": 1733770879,
      "dataset_name": "big-dataset",
      "description": "",
      "id": "2a9dd8bd-6c29-47c8-9ca4-427f73174881",
      "name": "132--after-concurrency"
    },
    {
      "created_at": 1732651609,
      "dataset_name": "Alec's Demo",
      "description": "",
      "id": "ab869ac0-e688-4abf-864d-cbce8dbbb1c5",
      "name": "test"
    },
    {
      "created_at": 1731966113,
      "dataset_name": "big-dataset",
      "description": "",
      "id": "64faf9fd-b263-4477-acfc-18f576befe4d",
      "name": "big-test"
    }
  ]
}

Customer Feedback

Completion Feedback

Completion feedback allows you to submit feedback for a specific completion in your project. Base URL: /api/v2/projects/<project-id>/completion-feedback

Create Completion Feedback

POST /id/<completion_id> Submits feedback for a completion, which can be used to track user satisfaction and improve model performance. Request Payload

Parameter	Type	Description	Required
feedback_attribute	`str` / `float` / `int` / `bool`	Key-value pairs of feedback attributes	Yes
freeplay_feedback	`str`	Value must be either `"positive"` or `"negative"`	Yes

Example Request

curl 'https://app.freeplay.ai/api/v2/projects/<project-id>/completion-feedback/id/707bc301-85e9-4f02-aa97-faba8cd7774a' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer TOKEN' \
  --data '{
    "feedback_attribute_1": "value",
    "feedback_attribute_2": 10,
    "freeplay_feedback": "positive"
  }'

Sample Response Payload:

{
  "message": "Feedback created successfully"
}

Errors:

400 Bad Request: Returned if the request payload contains invalid data types or invalid values for freeplay_feedback.

{
  "error": "Unexpected type for value. Values must be strings, numbers or booleans."
}

{
  "error": "Invalid value for 'freeplay_feedback': value. Must be 'positive' or 'negative'."
}

Trace Feedback

Trace feedback allows you to submit feedback for a specific trace in your project. Base URL: /api/v2/projects/<project-id>/trace-feedback

Create Trace Feedback

POST /id/<trace_id> Submits feedback for a trace, which can be used to track user satisfaction and improve model performance. Request Payload

Parameter	Type	Description	Required
feedback_attribute	`str` / `float` / `int` / `bool`	Key-value pairs of feedback attributes	Yes
freeplay_feedback	`str`	Value must be either `"positive"` or `"negative"`	Yes

Example Request

curl 'https://app.freeplay.ai/api/v2/projects/<project-id>/trace-feedback/id/<trace-id>' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer TOKEN' \
  --data '{
    "feedback_attribute_1": "value",
    "feedback_attribute_2": 10,
    "freeplay_feedback": "positive"
  }'

Sample Response Payload:

{
  "message": "Feedback created successfully"
}

Errors:

400 Bad Request: Returned if the request payload contains invalid data types or invalid values for freeplay_feedback.

{
  "error": "Unexpected type for value. Values must be strings, numbers or booleans."
}

{
  "error": "Invalid value for 'freeplay_feedback': value. Must be 'positive' or 'negative'."
}

Datasets

Base URL: /api/v2/projects/<project-id>/datasets

Upload Dataset Test Cases

By ID

POST /id/<dataset-id>/test-cases Uploads test cases to an existing dataset.

Maximum of 100 test cases per request.

curl --location 'https://app.freeplay.ai/api/v2/projects/5688ebaf-7f22-4d5d-b9bb-bc715c8faabb/datasets/id/e7bc4931-2cd4-4345-88c1-9547fdb62f94/test-cases' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer TOKEN' \
  --data '{
    "examples": [
      {
        "inputs": {
          "input_one": "value one a",
          "input_two": "value two a"
        },
        "output": "response one"
      },
      {
        "inputs": {
          "input_one": "value one a",
          "input_two": "value two b"
        },
        "output": "response two"
      }
    ]
  }'

Retrieve Dataset Metadata

By Name

GET /name/<dataset-name>

curl --location 'https://app.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/datasets/name/Sample' \
  --header 'Authorization: Bearer TOKEN'

Sample Response

{
  "description": "This is a test description",
  "id": "4995f471-7b3f-4719-b7cc-f211a84445eb",
  "name": "Sample"
}

By ID

GET /id/<dataset-id>

curl --location 'https://app.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/datasets/id/4995f471-7b3f-4719-b7cc-f211a84445eb' \
  --header 'Authorization: Bearer TOKEN'

Sample Response

{
  "description": "This is a test description",
  "id": "4995f471-7b3f-4719-b7cc-f211a84445eb",
  "name": "Sample"
}

Retrieve Dataset Test Cases

By Name

GET /name/<dataset-name>/test-cases

curl --location 'https://app.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/datasets/name/Sample/test-cases' \
  --header 'Authorization: Bearer TOKEN'

Sample Response

[
  {
    "history": null,
    "id": "115f950c-ca48-45c3-a372-1ef84cb9da44",
    "output": "Bullgle",
    "values": {
      "breedA": "Beagle",
      "breedB": "Bulldog"
    }
  },
  {
    "history": null,
    "id": "2cf1f967-8fc6-4ed7-a14e-97ffb1a3fce3",
    "output": "Porgi",
    "values": {
      "breedA": "Pug",
      "breedB": "Corgi"
    }
  },
  {
    "history": null,
    "id": "8ab1d78c-7e7d-4366-ac05-963780084aa0",
    "output": "Greatador",
    "values": {
      "breedA": "Labrador",
      "breedB": "Great Dane"
    }
  }
]

By ID

GET /id/<dataset-id>/test-cases

curl --location 'https://app.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/datasets/id/4995f471-7b3f-4719-b7cc-f211a84445eb/test-cases' \
  --header 'Authorization: Bearer TOKEN'

Sample Response

[
  {
    "history": null,
    "id": "115f950c-ca48-45c3-a372-1ef84cb9da44",
    "output": "Bullgle",
    "values": {
      "breedA": "Beagle",
      "breedB": "Bulldog"
    }
  },
  {
    "history": null,
    "id": "2cf1f967-8fc6-4ed7-a14e-97ffb1a3fce3",
    "output": "Porgi",
    "values": {
      "breedA": "Pug",
      "breedB": "Corgi"
    }
  },
  {
    "history": null,
    "id": "8ab1d78c-7e7d-4366-ac05-963780084aa0",
    "output": "Greatador",
    "values": {
      "breedA": "Labrador",
      "breedB": "Great Dane"
    }
  }
]

Completions

Base URL: /api/v2/projects/<project-id>/completions

Retrieve Completion Summary Statistics

Returns summary statistics for completions within a specified date range.

Statistics are served in a maximum date range of 30 days.

Request Payload

Parameter	Type	Description	Required
to_date	`str`	Latest date for which sessions will be returned (exclusive). Format: `YYYY-MM-DD`	No (Default: Today’s date)
from_date	`str`	Earliest date for which sessions will be returned (inclusive). Format: `YYYY-MM-DD`	No (Default: 7 days ago)

Validation Errors 400 Bad Request

{ "message": "Dates must be in YYYY-MM-DD format" }

{ "message": "To date must be greater than from date" }

{ "message": "Date range cannot exceed 30 days" }

Aggregate Statistics

POST /statistics Returns aggregated statistics across all prompt templates in the project.

curl --location 'https://app.freeplay.ai/api/v2/projects/45e20940-322a-4cb8-810f-3934a56aa5e2/completions/statistics' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer TOKEN' \
  --data '{
    "to_date": "2025-01-15",
    "from_date": "2025-01-10"
  }'

Sample Response

{
  "summary_info": {
    "query-rewrite": {},
    "rag-directions": {},
    "rag-qa": {
      "2025-01-10": {
        "auto_evals": {
          "Answer Accuracy": {
            "4": 4,
            "5": 12
          },
          "Answer Faithfulness": {
            "no": 1,
            "yes": 1
          },
          "Context Relevance": {
            "4": 5,
            "5": 10
          }
        },
        "client_evals": {},
        "human_evals": {}
      },
      "2025-01-11": {
        "auto_evals": {
          "Answer Accuracy": {
            "4": 2,
            "5": 12
          },
          "Answer Faithfulness": {
            "yes": 2
          },
          "Context Relevance": {
            "4": 4,
            "5": 12
          }
        },
        "client_evals": {},
        "human_evals": {}
      },
      "2025-01-12": {
        "auto_evals": {
          "Answer Accuracy": {
            "4": 2,
            "5": 14
          },
          "Answer Faithfulness": {
            "yes": 2
          },
          "Context Relevance": {
            "4": 2,
            "5": 10
          }
        },
        "client_evals": {},
        "human_evals": {}
      },
      "2025-01-13": {
        "auto_evals": {
          "Answer Accuracy": {
            "4": 3,
            "5": 10
          },
          "Context Relevance": {
            "4": 5,
            "5": 6
          }
        },
        "client_evals": {},
        "human_evals": {}
      },
      "2025-01-14": {
        "auto_evals": {
          "Answer Accuracy": {
            "5": 13
          },
          "Context Relevance": {
            "4": 1,
            "5": 7
          }
        },
        "client_evals": {},
        "human_evals": {}
      }
    },
    "rag-qa-common-elements": {},
    "rag-qa-structured": {},
    "rag-query": {}
  }
}

By Prompt Template ID

POST /statistics/<prompt-template-id> Returns statistics for completions associated with a specific prompt template.

curl --location 'https://app.freeplay.ai/api/v2/projects/45e20940-322a-4cb8-810f-3934a56aa5e2/completions/statistics/0053c449-c736-4557-a3d4-bf3b852d0604' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer TOKEN' \
  --data '{
    "to_date": "2025-01-15",
    "from_date": "2025-01-10"
  }'

Sample Response

{
  "summary_info": {
    "rag-qa": {
      "2025-01-10": {
        "auto_evals": {
          "Answer Accuracy": {
            "4": 4,
            "5": 12
          },
          "Answer Faithfulness": {
            "no": 1,
            "yes": 1
          },
          "Context Relevance": {
            "4": 5,
            "5": 10
          }
        },
        "client_evals": {},
        "human_evals": {}
      },
      "2025-01-11": {
        "auto_evals": {
          "Answer Accuracy": {
            "4": 2,
            "5": 12
          },
          "Answer Faithfulness": {
            "yes": 2
          },
          "Context Relevance": {
            "4": 4,
            "5": 12
          }
        },
        "client_evals": {},
        "human_evals": {}
      },
      "2025-01-12": {
        "auto_evals": {
          "Answer Accuracy": {
            "4": 2,
            "5": 14
          },
          "Answer Faithfulness": {
            "yes": 2
          },
          "Context Relevance": {
            "4": 2,
            "5": 10
          }
        },
        "client_evals": {},
        "human_evals": {}
      },
      "2025-01-13": {
        "auto_evals": {
          "Answer Accuracy": {
            "4": 3,
            "5": 10
          },
          "Context Relevance": {
            "4": 5,
            "5": 6
          }
        },
        "client_evals": {},
        "human_evals": {}
      },
      "2025-01-14": {
        "auto_evals": {
          "Answer Accuracy": {
            "5": 13
          },
          "Context Relevance": {
            "4": 1,
            "5": 7
          }
        },
        "client_evals": {},
        "human_evals": {}
      }
    }
  }
}

Getting started

Quick start

Core Concepts

AI Framework integrations

Practical Guides

Recipes

Security & Compliance

Resources

​API Reference

​Authentication

​Custom Domain

​API Keys

​API Errors

​Common Error Codes

​400 Bad Request

​401 Unauthorized

​404 Not Found

​500 Server Error

​Projects

​List All Projects

​Prompt Templates

​Retrieve a Prompt Template by Name

​Retrieve a Formatted Prompt

​Retrieve a Bound Prompt

​Retrieve a Raw Prompt Template

​Retrieve All Prompt Templates in an Environment

​Retrieve a Prompt Template by Version ID

​Create a Prompt Template Version

​Example Request

​Validation & Errors

​Update Environments for a Prompt Template Version

​Validation & Errors

​Agents

​List Agents

​Sessions

​Record a Completion

​Retrieve Existing Sessions/Completions

​Additional Filtering Examples

​Delete a Session

​Test Runs

​Create a Test Run

​Retrieve Test Run Results

​List Test Runs

​Customer Feedback

​Completion Feedback

​Create Completion Feedback

​Trace Feedback

​Create Trace Feedback

​Datasets

​Upload Dataset Test Cases

​By ID

​Retrieve Dataset Metadata

​By Name

​By ID

​Retrieve Dataset Test Cases

​By Name

​By ID

​Completions

​Retrieve Completion Summary Statistics

​Aggregate Statistics

​By Prompt Template ID

API Reference

Authentication

Custom Domain

API Keys

API Errors

Common Error Codes

400 Bad Request

401 Unauthorized

404 Not Found

500 Server Error

Projects

List All Projects

Prompt Templates

Retrieve a Prompt Template by Name

Retrieve a Formatted Prompt

Retrieve a Bound Prompt

Retrieve a Raw Prompt Template

Retrieve All Prompt Templates in an Environment

Retrieve a Prompt Template by Version ID

Create a Prompt Template Version

Example Request

Validation & Errors

Update Environments for a Prompt Template Version

Validation & Errors

Agents

List Agents

Sessions

Record a Completion

Retrieve Existing Sessions/Completions

Additional Filtering Examples

Delete a Session

Test Runs

Create a Test Run

Retrieve Test Run Results

List Test Runs

Customer Feedback

Completion Feedback

Create Completion Feedback

Trace Feedback

Create Trace Feedback

Datasets

Upload Dataset Test Cases

By ID

Retrieve Dataset Metadata

By Name

By ID

Retrieve Dataset Test Cases

By Name

By ID

Completions

Retrieve Completion Summary Statistics

Aggregate Statistics

By Prompt Template ID