Guides

API Reference

Suggest Edits

Authentication

Custom Domain

Your API root will be the URL from your Freeplay instance plus the suffix /api/v2/

For example https://acme.freeplay.ai/api/v2

API Keys

Freeplay authenticates your API request using your API Key which can be managed through the Freeplay application at https://acme.freeplay.ai/settings/api-access

To authenticate, provide an API key in the Authorization header

Authorization: Bearer {freeplay_api_key}

API Errors

When using the Freeplay HTTP API, you may encounter various error responses. We use standard HTTP error codes, such as 200 for success, 400 for bad requests, 401 for authorization errors, and 500 for server errors. Most errors will also include an HTTP body, encoded in JSON, describing what went wrong and how to fix the issue.

It is not helpful to retry errors with codes in the 400-499 range, because they are typically client errors and will not be fixed by future requests.

On the other hand, retrying requests that resulted in 500 error codes will sometimes work, for example in the case that a request was sent to an instance that was terminated or during a transient network issue. We recommend retrying requests that received a 500 response up to three times, after at least a 5 second delay.

Here's a list of common errors, their status codes, and explanations:

400 Bad request

This means that the request was malformed or had bad data. This could happen if the request is not proper JSON or if it contains invalid data.

{"message": "Session ID 456 is not a valid uuid4 format."}

401 Unauthorized

This response will be returned if an API request does not contain the correct bearer authentication with a valid API key.

{"message": "Requests must be authenticated with a valid API key."}

404 Not found

This response indicates one of the IDs passed in the request is invalid, and the object does not exist. This can happen if the referenced object was deleted, or if the caller does not have access to the object.

{"message": "Project not found"}

500 Server error

This response indicates an error in the Freeplay server. Our team is notified for issues of this type via PagerDuty. Sometimes these are transient issues that can be fixed by retrying the request after a delay.

{"message": "An unexpected error occurred. Please try again later."}

When encountering any of these errors, check your request parameters and payload to ensure they match the API specifications. If issues persist, contact Freeplay support for further assistance.

Prompt Templates

A prompt template in the Freeplay platform is a pre-defined structure for creating prompts that are used to interact with LLMs.

Base URL: /api/v2/projects/<project-id>/prompt-templates

Retrieve a Prompt Template by Name

GET /name/<template-name>

You can retrieve a Prompt Template in one of three forms: Raw, Bound, or Formatted. Look at sample response payloads below for field details.

Raw will give you the Prompt Template without Input Variable values inserted

Bound will give you the Prompt Template with Input Variable values inserted and with content in a consistent format. Pass your Input Variables in the request body to get a Bound Prompt.

Formatted (most common) will give you the Prompt Template with Input Variable values inserted and with content formatted according to your LLM provider. Pass format = true as a query parameter and pass your Input Variables in the request body to get a Formatted Prompt

Query Parameters

ParameterTypeDescriptionRequired
environmentstrThe environment tag of the prompt you want to fetchNo
Default: latest
formatbooleanIndicates whether to return a Formatted Prompt or a Bound PromptNo

Retrieve a Formatted Prompt

curl 'https://example.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/prompt-templates/name/album_bot?environment=prod&format=true' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer TOKEN' \
--data '{
    "pop_star": "justin beiber"
}'

# SAMPLE RESPONSE PAYLOAD
{
    "format_version": 2,
    "formatted_content": [
        {
            "content": "You are a helpful bot that generates album names\nGenerate a two word album name in the style of justin beiber",
            "role": "user"
        }
    ],
    "formatted_tool_schema": [
        {
        "function": {
            "description": "Get weather of a location",
            "name": "weather_of_location",
            "parameters": {
            "properties": {
                "location": {
                "description": "Location to get the weather for",
                "type": "string"
                }
            },
            "required": [
                "location"
            ],
            "type": "object"
            }
        },
        "type": "function"
        }
    ],
    "metadata": {
        "flavor": "openai_chat",
        "model": "gpt-3.5-turbo-0125",
        "params": {
            "max_tokens": 100,
            "temperature": 0.2
        },
        "provider": "openai",
        "provider_info": {}
    },
    "prompt_template_id": "5cccc8e6-b163-4094-8bd2-90030f151ec8",
    "prompt_template_name": "album_bot",
    "prompt_template_version_id": "f503c15e-2f0f-4ce4-b443-4c87d0b6435d",
    "system_content": null
}

Retrieve a Bound Prompt

curl 'https://example.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/prompt-templates/name/album_bot?environment=prod' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer TOKEN' \
--data '{
    "pop_star": "justin beiber"
}'

# SAMPLE RESPONSE PAYLOAD
{
    "bound_content": [
        {
            "content": "You are a helpful bot that generates album names\nGenerate a two word album name in the style of justin beiber",
            "role": "user"
        }
    ],
    "format_version": 2,
    "metadata": {
        "flavor": "openai_chat",
        "model": "gpt-3.5-turbo-0125",
        "params": {
            "max_tokens": 100,
            "temperature": 0.2
        },
        "provider": "openai",
        "provider_info": {}
    },
    "prompt_template_id": "5cccc8e6-b163-4094-8bd2-90030f151ec8",
    "prompt_template_name": "album_bot",
    "prompt_template_version_id": "f503c15e-2f0f-4ce4-b443-4c87d0b6435d",
    "tool_schema": [
        {
        "description": "Get weather of a location",
        "name": "weather_of_location",
        "parameters": {
            "additionalProperties": false,
            "properties": {
            "location": {
                "description": "Location to get the weather for",
                "type": "string"
            }
            },
            "required": [
            "location"
            ],
            "type": "object"
        }
        }
    ]
}

Retrieve a Raw Prompt Template

curl 'https://example.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/prompt-templates/name/album_bot?environment=prod' \
--header 'Authorization: Bearer TOKEN' \

# SAMPLE RESPONSE PAYLOAD
{
    "content": [
        {
            "content": "You are a helpful bot that generates album names\nGenerate a two word album name in the style of {{pop_star}}",
            "role": "user"
        }
    ],
    "format_version": 2,
    "metadata": {
        "flavor": "openai_chat",
        "model": "gpt-3.5-turbo-0125",
        "params": {
            "max_tokens": 100,
            "temperature": 0.2
        },
        "provider": "openai",
        "provider_info": {}
    },
    "prompt_template_id": "5cccc8e6-b163-4094-8bd2-90030f151ec8",
    "prompt_template_name": "album_bot",
    "prompt_template_version_id": "f503c15e-2f0f-4ce4-b443-4c87d0b6435d",
    "tool_schema": [
        {
        "description": "Get weather of a location",
        "name": "weather_of_location",
        "parameters": {
            "additionalProperties": false,
            "properties": {
            "location": {
                "description": "Location to get the weather for",
                "type": "string"
            }
            },
            "required": [
            "location"
            ],
            "type": "object"
        }
        }
    ]
}

Retrieve all Prompt Templates in an Environment

GET /all/<environment-name>

curl 'https://example.freeplay.ai/api/v2/projects/<project-id>/prompt-templates/all/latest' \
--header 'Authorization: Bearer TOKEN' \

# Sample Response Payload
{
    "prompt_templates": [
        {
            "content": [
                {
                    "content": "You are a helpful bot that generates album names\nRespond with only the album name",
                    "role": "system"
                },
                {
                    "content": "Generate a two word album name in the style of {{pop_star}}",
                    "role": "user"
                }
            ],
            "format_version": 2,
            "metadata": {
                "flavor": "anthropic_chat",
                "model": "claude-3-opus-20240229",
                "params": {
                    "max_tokens": 256,
                    "temperature": 0.5
                },
                "provider": "anthropic",
                "provider_info": {}
            },
            "prompt_template_id": "5cccc8e6-b163-4094-8bd2-90030f151ec8",
            "prompt_template_name": "album_bot",
            "prompt_template_version_id": "eb934c9f-ee1e-42be-a9cb-d193f5d3aa54"
        },
        {
            "content": [
                {
                    "content": "You are a news article summarization agent. Your job is to find the big idea in an article, distill it into compelling content & write a catchy summary that people want to click. You will be provided with the article title, source url and the article text\nThe scraped source content of the URL for the article is provided. ",
                    "role": "system"
                },
                {
                    "content": "Here is the scraped content from the web page:\n--------\n{{article_title}}\n{{article_text}}\n{{source_url}}\n--------\n\nSome additional instructions follow. Please follow these carefully.\n\nThere may be user comments at the bottom or other superfluous information included from the scraped content. Ignore those and focus on the body of the article. \n\nAt the end of each response, after a newline always include the original URL for the article so people can read the article. \n\nWhen sharing the URL, include it after two line breaks and use this format starting with \"Read more\" and stripping the http:// at the start of the url: Read more: www.example.com\nGiven this article, respond with a Tweet of no more than 200 characters. Never include hashtags. A relevant emoji is ok. Do not include quotes around any content.\n\nAfter a newline, include the original URL for the article so people can read the article.",
                    "role": "user"
                }
            ],
            "format_version": 2,
            "metadata": {
                "flavor": "openai_chat",
                "model": "gpt-3.5-turbo-16k-0613",
                "params": {
                    "max_tokens": 2000,
                    "temperature": 0.5
                },
                "provider": "openai",
                "provider_info": {}
            },
            "prompt_template_id": "9daa7233-16d3-48b3-a739-ddba66742b08",
            "prompt_template_name": "article_summarizer",
            "prompt_template_version_id": "9f9f9bfe-ed53-449d-a13c-5f88198c0f8e"
        }
}

Retrieve a Prompt Template by Version Id

Retrieve a Prompt Template using a specific Version Id. This can be useful if you want to pin your code to a specific prompt version regardless of actions taken in the Freeplay app

GET /id/<template-id>/versions/<template-version-id>

The same Raw, Bound, and Formatted options exist when retrieving by version id as they do when retrieving by name

curl 'https://example.freeplay.ai//api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/prompt-templates/id/0053c449-c736-4557-a3d4-bf3b852d0604/versions/42d301b2-cdc3-48d5-829b-b892561017ff' \
--header 'Authorization: Bearer TOKEN'

# Sample response payload
{
    "content": [
        {
            "content": "You are a technical customer support agent for a product called freeplay. Your job is to help users answer questions about freeplay.\nYou will often be provided with information that will help you answer the question. If the information is relevant then use it to answer the question and end your response with \"See <source> for more details\" where <source> is a valid HTML formatted hyperlink to the source URL.\n\nIf the information provided is not relevant then ignore it.\n\nUser Query: {{question}}\n----\nSupporting Information: {{supporting_information}}",
            "role": "user"
        }
    ],
    "format_version": 2,
    "metadata": {
        "flavor": "openai_chat",
        "model": "gpt-3.5-turbo-16k-0613",
        "params": {
            "max_tokens": 2000,
            "temperature": 0.5
        },
        "provider": "openai",
        "provider_info": {}
    },
    "prompt_template_id": "0053c449-c736-4557-a3d4-bf3b852d0604",
    "prompt_template_name": "rag-qa",
    "prompt_template_version_id": "42d301b2-cdc3-48d5-829b-b892561017ff"
}

Sessions

Sessions are a collection of 1 or more LLM completions — enabling you to tie completions together in whatever logical grouping makes sense for your application (multi-turn chat, chains, agent flows, etc.).

Base URL: /api/v2/projects/<project-id>/sessions

Record a Completion

POST /<session-id>/completions

Request Payload

ParameterTypeDescriptionRequired
messageslist[{"role": Literal["user", "assistant"], "content": str}]List of messages send to the LLM.

Note for Anthropic: messages must exclude message fields other than role and content.		Yes
tool_schemalist[{ name: str, description: str, parameters: dict[str, any] }]Tool schema used with the completionNo
inputsdict[str, any]Input variables used to build the formatted prompt. Values must not be null.Yes
session_infocustom_metatdata: dict[str, str]Information to be associated with the SessionNo
prompt_infoprompt_template_version_id: UUID environment: strInformation associated with the Prompt TemplateYes
call_infostart_time: float
end_time: float
model: float
provider: float
provider_info: dict[str, any]
llm_parameters: dict[str, any]		Information associated with the LLM callNo
test_run_infotest_run_id: UUID test_case_id: UUIDInformation for the associated Test RunNo
completion_idUUIDThe UUIDv4 that should be used as the ID for the completion. Valid for sending completion feedback.No
curl 'https://example.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/sessions/f503c15e-2f0f-4ce4-b443-4c87d0b6435d/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer TOKEN' \
--data '{
  "messages": [
    {
      "content": "You are a helpful bot that generates album names\nGenerate a two word album name in the style of Taylor Swift",
      "role": "user"
    },
    {
      "role": "assistant",
      "content": "\"Rainy Melodies\""
    }
  ],
  "inputs": {
    "pop_star": "Taylor Swift"
  },
  "prompt_info": {
    "prompt_template_version_id": "f503c15e-2f0f-4ce4-b443-4c87d0b6435d",
    "environment": "prod"
  }
}
'

# SAMPLE RESPONSE PAYLOAD
{
    "completion_id": "707bc301-85e9-4f02-aa97-faba8cd7774a"
}

Record an LLM Interaction End to End

import requests
from dotenv import load_dotenv
import os
import json
import uuid

load_dotenv("../.env")
project_id = os.getenv("FREEPLAY_PROJECT_ID")
freeplay_api_root = f"https://example.freeplay.ai/api/v2/projects/{project_id}"
openai_api_url = "https://api.openai.com/v1/chat/completions" 

## PROMPT ##
# fetch the prompt
prompt_name = "album_bot"
input_variables = {"pop_star": "Taylor Swift"}
prompt_req = requests.post(
    url=f"{freeplay_api_root}/prompt-templates/name/{prompt_name}",
    headers={
        "Authorization" : f"Bearer {os.getenv('FREEPLAY_KEY')}"
    },
    params={
        "environment": "prod",
        "format": "true"
    },
    data=json.dumps(input_variables)
)
formatted_prompt = prompt_req.json()

## LLM ##
completion_payload = {
        "model": formatted_prompt["metadata"]["model"],
        "messages": formatted_prompt["formatted_content"],
        **formatted_prompt["metadata"]["params"]
}

completion_req = requests.post(
    url=openai_api_url,
    headers={
        "Authorization" : f"Bearer {os.getenv('OPENAI_API_KEY')}",
        "Content-Type": "application/json"
    },
    data=json.dumps(completion_payload)
)
response_message = completion_req.json()['choices'][0]['message']

## RECORD ##
# record the interaction to freeplay
# add the LLM response to the message queue
messages = formatted_prompt["formatted_content"]
messages.append(response_message)

# construct the payload
record_payload = {
    "messages": messages,
    "inputs": input_variables,
    "prompt_info": {
        "prompt_template_version_id": formatted_prompt["prompt_template_version_id"],
        "environment": "prod"
    }
}

# create a session id
session_id = str(uuid.uuid4())

# record to freeplay
requests.post(
    url=f"{freeplay_api_root}/sessions/{session_id}/completions",
    headers={
        "Authorization": f"Bearer {os.getenv('FREEPLAY_KEY')}",
        "Content-Type": "application/json"
    },
    data = json.dumps(record_payload)
)

Retrieve Existing Sessions/Completions

GET /

Retrieve sessions, including their metadata and completions, starting from most recent first.

Parameters

ParameterTypeDescriptionRequired
pageintThe page number of the sessions to retrieve. Maximum page size is 100.No
Default: 1
page_sizeintThe number of sessions to include in the request.No
Default: 10
test_liststrThe name of a test list. Only sessions from this test list will be returned.No
from_datestrA date that is the earliest for which sessions will be returned (inclusive). Must be in the format YYYY-MM-DD.No
to_datestrA date that is the latest for which sessions will be returned (exclusive). Must be in the format YYYY-MM-DD. No
curl --header "Authorization: Bearer $FREEPLAY_API_KEY" "https://example.freeplay.ai/api/v2/projects/$FREEPLAY_PROJECT_ID/sessions"

# Sample Response Payload
[
  {
    "custom_metadata": {},
    "messages": [
      {
        "client_evaluation_results": [],
        "completion_id": "0202ae57-1098-4d3e-94f7-1fbb034fbed5",
        "customer_feedback": {},
        "environment": "latest",
        "evaluation_results": [],
        "format_version": 3,
        "input_variables": {
          "question": "How do I bundle prompts so my system is reliable?"
        },
        "message_type": "completion",
        "model_name": "claude-2.1",
        "prompt": [
          {
            "content": "You are a technical customer support agent for a product called freeplay.",
            "role": "system"
          },
          {
            "content": "How do I bundle prompts so my system is reliable?",
            "role": "user"
          }
        ],
        "prompt_template_name": "anthropic",
        "provider_name": "anthropic",
        "response": "You bundle prompts by downloading them with the python SDK, packaging them with your application, and using the FilesystemTemplateResolver when using the Freeplay SDK.",
        "trace_id": null
      }
    ],
    "session_id": "85d7d393-4e85-4664-8598-5dd91dc75b5b",
    "start_time": "2024-07-05T14:33:02.721000"
  }
]

Delete a Session

DELETE /api/v2/projects/<project-id>/sessions/<session-id>

curl --request DELETE 'https://example.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/sessions/a903c15e-2f0f-4ce4-b743-4c87d0c64918' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer TOKEN'

Test Runs

Test Runs in Freeplay provide a structured way to generate batch tests for your language models using saved Datasets managed on the Freeplay server.

Base URL: /api/v2/projects/<project-id>/test-runs

Create a Test Run

POST /

Request Payload

ParametersTypeDescriptionRequired
dataset_namestrThe name of the dataset for which you want to create a Test RunYes
include_outputsbooleanDetermines if Outputs are returned for examplesNo
Default: True
test_run_namestrSet a name for the Test Run, to be displayed in the Freeplay appNo
test_run_descriptionstrSet a description for the Test Run, to be displayed in the Freeplay appNo
curl 'https://example.freeplay.ai/api/v2/projects/<project-id>/test-runs' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer TOKEN' \
--data '{
    "dataset_name": "Example Tests"
}'

# Sample Response Payload
{
    "test_cases": [
        {
            "output": null,
            "test_case_id": "91e60c9e-fbaa-4990-b4cc-7a8bd067f298",
            "variables": {
                "question": "How do test runs relate to test cases?",
                "supporting_information": "([SearchChunk(title='Test Runs', link='https://docs.freeplay.ai/docs/test-runs-with-freeplay', content='Test Runs')..."
            }
        },
        {
            "output": null,
            "test_case_id": "c45dbc01-a177-449b-ab66-bb5a469748fe",
            "variables": {
                "question": "Write me some typescript code to run a new Test Run in freeplay",
                "supporting_information": "[{\"id\":88,\"version\":5,\"score\":0.8668302,\"payload\":{\"description\":\"Ship better products with LLMs...."
            }
        }  
    ],
    "test_run_description": "",
    "test_run_id": "bd3eb06c-f93b-46a4-aa3b-d240789c8a06",
    "test_run_name": ""
}

Execute a Test Run end to end

import requests
from dotenv import load_dotenv
import os
import json
import uuid

load_dotenv("../.env")
project_id = "45e20940-322a-4cb8-810f-3934a56aa5e2"
freeplay_api_root = f"https://example.freeplay.ai/api/v2/projects/{project_id}"
openai_api_url = "https://api.openai.com/v1/chat/completions" 

# create the test run
test_run_req = requests.post(
    url= freeplay_api_root + "/test-runs",
    headers={
        "Authorization": f"Bearer {os.getenv('FREEPLAY_KEY')}",
        "Content-Type": "application/json"
    },
    data=json.dumps({
        "dataset_name": "Example Tests"
    })
)
test_run = test_run_req.json()

prompt_name = "rag-qa"
# loop over each test case in the dataset
for test_case in test_run["test_cases"]:
    # fetch the formatted prompt
    prompt_req = requests.post(
        url=f"{freeplay_api_root}/prompt-templates/name/{prompt_name}",
        headers={
            "Authorization" : f"Bearer {os.getenv('FREEPLAY_KEY')}"
        },
        params={
            "environment": "prod",
            "format": "true"
        },
        data=json.dumps(test_case['variables'])
    )
    formatted_prompt = prompt_req.json()

    # execute the completion
    completion_payload = {
        "model": formatted_prompt["metadata"]["model"],
        "messages": formatted_prompt["formatted_content"],
        **formatted_prompt["metadata"]["params"]
    }
    completion_req = requests.post(
        url=openai_api_url,
        headers={
            "Authorization" : f"Bearer {os.getenv('OPENAI_API_KEY')}",
            "Content-Type": "application/json"
        },
        data=json.dumps(completion_payload)
    )
    response_message = completion_req.json()['choices'][0]['message']

    # record to freeplay
    messages = formatted_prompt["formatted_content"]
    messages.append(response_message)

    record_payload = {
        "messages": messages,
        "inputs": test_case['variables'],
        "prompt_info": {
            "prompt_template_version_id": formatted_prompt["prompt_template_version_id"],
            "environment": "prod"
        },
        # include the test run info to link the session to the test run
        "test_run_info": {
            "test_run_id": test_run["test_run_id"],
            "test_case_id": test_case["test_case_id"]
        }
    }

    # create a session
    session_id = str(uuid.uuid4())
    record_res = requests.post(
        url=f"{freeplay_api_root}/sessions/{session_id}/completions",
        headers={
            "Authorization": f"Bearer {os.getenv('FREEPLAY_KEY')}",
            "Content-Type": "application/json"
        },
        data=json.dumps(record_payload)
    )

Retrieve Test Run Results

GET /id/<test-run-id>

curl 'https://example.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/test-runs/id/2a9dd8bd-6c29-47c8-9ca4-427f73174881' \
--header 'Authorization: Bearer TOKEN' 

# sample response
{
    "created_at": 1733770879,
    "description": "",
    "id": "2a9dd8bd-6c29-47c8-9ca4-427f73174881",
    "model_name": "gpt-4o-mini-2024-07-18",
    "name": "132--after-concurrency",
    "prompt_name": "rag-qa",
    "prompt_version": "gpt-4o-mini",
    "sessions_count": 132,
    "summary_statistics": {
        "auto_evaluation": {
            "Answer Accuracy": {
                "1": 104,
                "2": 9,
                "4": 9,
                "5": 10
            },
            "Answer Faithfulness": {
                "no": 126,
                "yes": 6
            },
            "Context Relevance": {
                "1": 116,
                "4": 2,
                "5": 14
            }
        },
        "client_evaluation": {},
        "human_evaluation": {}
    }
}

List Test Runs

GET /

Test Run will be returned in descending chronologic order. You can use pagination to get to test runs not in the most recent 100.

Parameters

ParameterTypeDescriptionRequired
pageintThe page number of the sessions to retrieve. Maximum page size is 100.No
Default: 1
page_sizeintThe number of sessions to include in the request.No
Default: 100
curl --location 'https://example.freeplay.ai/api/v2/projects/45e20940-322a-4cb8-810f-3934a56aa5e2/test-runs' \
--header 'Authorization: Bearer TOKEN'

# Sample Response
{
    "test_runs": [
        {
            "created_at": 1733770879,
            "dataset_name": "big-dataset",
            "description": "",
            "id": "2a9dd8bd-6c29-47c8-9ca4-427f73174881",
            "name": "132--after-concurrency"
        },
        {
            "created_at": 1732651609,
            "dataset_name": "Alec's Demo",
            "description": "",
            "id": "ab869ac0-e688-4abf-864d-cbce8dbbb1c5",
            "name": "test"
        },
        {
            "created_at": 1731966113,
            "dataset_name": "big-dataset",
            "description": "",
            "id": "64faf9fd-b263-4477-acfc-18f576befe4d",
            "name": "big-test"
        }
    ]
}

Customer Feedback

Completion Feedback

Completion Feedback allows you to submit feedback for a specific completion in your project.

Base URL: /api/v2/projects/<project-id>/completion-feedback

Create Completion Feedback

Endpoint:
POST /id/<completion_id>

Request Payload:

ParameterTypeDescriptionRequired
feedback_attributestr/float/int/boolKey-value pairs of feedback attributes.Yes
freeplay_feedbackstrValue must be either "positive" or "negative".Yes

Example cURL Request:

curl 'https://example.freeplay.ai/api/v2/completion-feedback/id/707bc301-85e9-4f02-aa97-faba8cd7774a' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer TOKEN' \
--data '{
  "feedback_attribute_1": "value",
  "feedback_attribute_2": 10,
  "freeplay_feedback": "positive"
}'

Sample Response Payload:

{
  "message": "Feedback created successfully"
}

Errors:

  • 400 Bad Request: Returned if the request payload contains invalid data types or invalid values for freeplay_feedback.
{
  "error": "Unexpected type for value. Values must be strings, numbers or booleans."
}

{
  "error": "Invalid value for 'freeplay_feedback': value. Must be 'positive' or 'negative'."
}

Trace Feedback

Trace Feedback allows you to submit feedback for a specific trace in your project.

Base URL: /api/v2/projects/<project-id>/trace-feedback

Create Trace Feedback

Endpoint:POST /id/<trace_id>

Request Payload:

ParameterTypeDescriptionRequired
feedback_attributestr/float/int/boolKey-value pairs of feedback attributes.Yes
freeplay_feedbackstrValue must be either "positive" or "negative".Yes

Example cURL Request:

curl 'https://example.freeplay.ai/api/v2/trace-feedback/id/0202ae57-1098-4d3e-94f7-1fbb034fbed5' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer TOKEN' \
--data '{
  "feedback_attribute_1": "value",
  "feedback_attribute_2": 10,
  "freeplay_feedback": "positive"
}'

Sample Response Payload:

{
  "message": "Feedback created successfully"
}

Errors:

  • 400 Bad Request: Returned if the request payload contains invalid data types or invalid values for freeplay_feedback.
{
  "error": "Unexpected type for value. Values must be strings, numbers or booleans."
}

{
  "error": "Invalid value for 'freeplay_feedback': value. Must be 'positive' or 'negative'."
}

Datasets

Base URL: /api/v2/projects/<project-id>/datasets

Upload Dataset Examples

By ID

POST/id/<dataset-id>/test-cases

curl --location 'https://example.freeplay.ai/api/v2/projects/5688ebaf-7f22-4d5d-b9bb-bc715c8faabb/datasets/id/e7bc4931-2cd4-4345-88c1-9547fdb62f94/test-cases' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer TOKEN' \
--data '{
    "examples": [
        {"inputs": {"input_one": "value one a", "input_two": "value two a"}, "output": "response one"},
        {"inputs": {"input_one": "value one a", "input_two": "value two b"}, "output": "response two"}
    ]
}
'

** Note limit of 100 examples per request

Retrieve Dataset Metadata

By Name

GET /name/<dataset-name>

curl --location 'https://example.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/datasets/name/Sample' \
--header 'Authorization: Bearer TOKEN'

# response
{
    "description": "This is a test description",
    "id": "4995f471-7b3f-4719-b7cc-f211a84445eb",
    "name": "Sample"
}

By ID

GET /id/<dataset-id>

curl --location 'https://example.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/datasets/id/4995f471-7b3f-4719-b7cc-f211a84445eb' \
--header 'Authorization: Bearer TOKEN'

{
    "description": "This is a test description",
    "id": "4995f471-7b3f-4719-b7cc-f211a84445eb",
    "name": "Sample"
}

Retrieve Dataset Examples

By Name

GET /name/<dataset-name>/test-cases

curl --location 'https://example.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/datasets/name/Sample/test-cases' \
--header 'Authorization: Bearer TOKEN'

[
    {
        "history": null,
        "id": "115f950c-ca48-45c3-a372-1ef84cb9da44",
        "output": "Bullgle",
        "values": {
            "breedA": "Beagle",
            "breedB": "Bulldog"
        }
    },
    {
        "history": null,
        "id": "2cf1f967-8fc6-4ed7-a14e-97ffb1a3fce3",
        "output": "Porgi",
        "values": {
            "breedA": "Pug",
            "breedB": "Corgi"
        }
    },
    {
        "history": null,
        "id": "8ab1d78c-7e7d-4366-ac05-963780084aa0",
        "output": "Greatador",
        "values": {
            "breedA": "Labrador",
            "breedB": "Great Dane"
        }
    }
]

By ID

GET /id/<dataset-id>/test-cases

curl --location 'https://example.freeplay.ai/api/v2/projects/8f93dd00-2eb5-4ba2-9354-86d5c6831dfd/datasets/id/4995f471-7b3f-4719-b7cc-f211a84445eb/test-cases' \
--header 'Authorization: Bearer TOKEN'

[
    {
        "history": null,
        "id": "115f950c-ca48-45c3-a372-1ef84cb9da44",
        "output": "Bullgle",
        "values": {
            "breedA": "Beagle",
            "breedB": "Bulldog"
        }
    },
    {
        "history": null,
        "id": "2cf1f967-8fc6-4ed7-a14e-97ffb1a3fce3",
        "output": "Porgi",
        "values": {
            "breedA": "Pug",
            "breedB": "Corgi"
        }
    },
    {
        "history": null,
        "id": "8ab1d78c-7e7d-4366-ac05-963780084aa0",
        "output": "Greatador",
        "values": {
            "breedA": "Labrador",
            "breedB": "Great Dane"
        }
    }
]

Completions

Base URL: /api/v2/projects/<project-id>/completions

Retrieve Completion Summary Statistics

Request Payload: Statistics are served in a maximum date range of 30 days

ParameterTypeDescriptionRequired
to_datestrA date that is the latest for which sessions will be returned (exclusive). Must be in the format YYYY-MM-DD.No - Default: Today's date
from_datestrA date that is the earliest for which sessions will be returned (inclusive). Must be in the format YYYY-MM-DD.No - Default: 7 days ago

Example errors:

  • 400 Bad Request: Returned if the request payload contains invalid date ranges
{"message": "Dates must be in YYYY-MM-DD format"}

{"message": "To date must be greater than from date"}

{"message": "Date range cannot exceed 30 days"}

Aggregate

POST: /statistics

curl --location 'https://example.freeplay.ai/api/v2/projects/45e20940-322a-4cb8-810f-3934a56aa5e2/completions/statistics' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer TOKEN' \
--data '{
    "to_date": "2025-01-15",
    "from_date": "2025-01-10"
}
'

# Sample Response
{
    "summary_info": {
        "query-rewrite": {},
        "rag-directions": {},
        "rag-qa": {
            "2025-01-10": {
                "auto_evals": {
                    "Answer Accuracy": {
                        "4": 4,
                        "5": 12
                    },
                    "Answer Faithfulness": {
                        "no": 1,
                        "yes": 1
                    },
                    "Context Relevance": {
                        "4": 5,
                        "5": 10
                    }
                },
                "client_evals": {},
                "human_evals": {}
            },
            "2025-01-11": {
                "auto_evals": {
                    "Answer Accuracy": {
                        "4": 2,
                        "5": 12
                    },
                    "Answer Faithfulness": {
                        "yes": 2
                    },
                    "Context Relevance": {
                        "4": 4,
                        "5": 12
                    }
                },
                "client_evals": {},
                "human_evals": {}
            },
            "2025-01-12": {
                "auto_evals": {
                    "Answer Accuracy": {
                        "4": 2,
                        "5": 14
                    },
                    "Answer Faithfulness": {
                        "yes": 2
                    },
                    "Context Relevance": {
                        "4": 2,
                        "5": 10
                    }
                },
                "client_evals": {},
                "human_evals": {}
            },
            "2025-01-13": {
                "auto_evals": {
                    "Answer Accuracy": {
                        "4": 3,
                        "5": 10
                    },
                    "Context Relevance": {
                        "4": 5,
                        "5": 6
                    }
                },
                "client_evals": {},
                "human_evals": {}
            },
            "2025-01-14": {
                "auto_evals": {
                    "Answer Accuracy": {
                        "5": 13
                    },
                    "Context Relevance": {
                        "4": 1,
                        "5": 7
                    }
                },
                "client_evals": {},
                "human_evals": {}
            }
        },
        "rag-qa-common-elements": {},
        "rag-qa-structured": {},
        "rag-query": {}
    }
}

By Prompt Template ID

POST: /statistics/<prompt-template-id>

curl --location 'https://example.freeplay.ai/api/v2/projects/45e20940-322a-4cb8-810f-3934a56aa5e2/completions/statistics/0053c449-c736-4557-a3d4-bf3b852d0604' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer TOKEN' \
--data '{
    "to_date": "2025-01-15",
    "from_date": "2025-01-10"
}
'

# Sample Response
{
    "summary_info": {
        "rag-qa": {
            "2025-01-10": {
                "auto_evals": {
                    "Answer Accuracy": {
                        "4": 4,
                        "5": 12
                    },
                    "Answer Faithfulness": {
                        "no": 1,
                        "yes": 1
                    },
                    "Context Relevance": {
                        "4": 5,
                        "5": 10
                    }
                },
                "client_evals": {},
                "human_evals": {}
            },
            "2025-01-11": {
                "auto_evals": {
                    "Answer Accuracy": {
                        "4": 2,
                        "5": 12
                    },
                    "Answer Faithfulness": {
                        "yes": 2
                    },
                    "Context Relevance": {
                        "4": 4,
                        "5": 12
                    }
                },
                "client_evals": {},
                "human_evals": {}
            },
            "2025-01-12": {
                "auto_evals": {
                    "Answer Accuracy": {
                        "4": 2,
                        "5": 14
                    },
                    "Answer Faithfulness": {
                        "yes": 2
                    },
                    "Context Relevance": {
                        "4": 2,
                        "5": 10
                    }
                },
                "client_evals": {},
                "human_evals": {}
            },
            "2025-01-13": {
                "auto_evals": {
                    "Answer Accuracy": {
                        "4": 3,
                        "5": 10
                    },
                    "Context Relevance": {
                        "4": 5,
                        "5": 6
                    }
                },
                "client_evals": {},
                "human_evals": {}
            },
            "2025-01-14": {
                "auto_evals": {
                    "Answer Accuracy": {
                        "5": 13
                    },
                    "Context Relevance": {
                        "4": 1,
                        "5": 7
                    }
                },
                "client_evals": {},
                "human_evals": {}
            }
        }
    }
}

Updated 4 days ago