💡
Getting Freeplay set up for the first time
This page walks through the basic process to get Freeplay set up and recording live data from your code.

Create a Project

Create & deploy a Prompt Template

Create Evaluation criteria

Integrate with the Freeplay SDKs or API to record Sessions

View Sessions

Before you get started

To get the most out of this guide, you'll first need:

A Freeplay instance. Your instance will be available at <subdomain>.freeplay.ai. If you don't have one yet, please reach out to us at [email protected] and we’ll get you set up. 🙌
An LLM provider API key. You'll need an API key for one or more of our supported providers (e.g. OpenAI, Anthropic).

Create a new Project

The first step is to create a new Project in the dashboard.

A "Project" equates to a prompt or collection of prompts that you want to use to power a feature in your application. For unrelated prompts/separate features in your application, create separate Projects.

Log into your Freeplay instance

Your instance will be available at <subdomain>.freeplay.ai. Enter the email address associated with your Freeplay account.

Set up a Project

Click New Project in the top right corner of your dashboard. Give your Project a unique name. You can use any name you want, but we suggest that you use a name related to the feature that you're building: e.g. Company Name Generator.

Create a Prompt Template

Freeplay provides a native prompt management platform enabling you to

Manage and version prompts and model configurations in app
Edit prompts in an interactive playground with real test cases
Define variables and control flow using mustache syntax
Deploy different prompt versions across different environments

Getting your prompts out of txt files managed in Git and into a native prompt management platform allows you to better track changes over time and opens the aperture of folks who can contribute to the prompt engineering process. Subject matter experts and product people can have a lot to contribute!

Craft your Prompt Template

After creating your Project you'll be prompted to create a new Prompt Template.
First select the model you want to use and associated parameters. For this example, we've chosen gpt-4from OpenAI with the default settings.

Then you can draft the first version of your prompt template. Prompt Templates are meant to be a reusable template. Content that is dynamically provided by your application code at run time will be denoted using Input Variables.

In this case we are creating a Prompt Template for a RAG application so we will give high level directions in the System message and then create Input Variables as placeholders for Question and Supporting Information which will be injected at run time in our application.

This prompt editor is a fully interactive playground meaning you can run your prompt with different wording or model configurations and compare directly in the UI as you're crafting your Prompt Template.

For example we could compare OpenAI and Anthropic side by side in the prompt editor across real input data.

Save and Deploy

Once you settle on the right Prompt Template you can save it with a Version Name and Description. You can then optionally deploy it to one or more of your environments.

Create Evaluation Criteria

Note this step is optional, we recommend configuring evaluation criteria to realize the full value of the platform but you can come back and do this at a later time.

Freeplay provides an end-to-end evaluation platform for evaluating and monitoring your LLM applications. It all starts with defining Evaluation Criteria associated with your Prompt Template. At Freeplay, we believe enterprise use cases require customizability which is why we have made our evaluation criteria fully configurable. Variable syntax is extended to evaluation criteria allowing you to target specific parts of your prompt for evaluation.

Configuring Evaluation Criteria

You can get a full breakdown of creating Evaluation Criteria here.

Being that this is a RAG application, we want evaluation criteria that independently target
a) The Retrieval Step (Context Relevance)
b) The Generation Step (Answer Faithfulness)
c) The final Output (Answer Relevance, Answer Correctness)

This allows us to understand which components of our pipeline are working well and which need some tweaking.

Integrate Freeplay with your Application Code

Freeplay offers a flexible SDK that is optimized for developer control, keeping core application code fully in your hands without wrappers or proxies. Simply fetch and format your prompt from Freeplay, make your LLM call, and record the results back to Freeplay.

Install the Freeplay SDK

Freeplay offers native SDKs for Python, Node.js, and Java (for use with any JVM language). Don't see an SDK you need? Please reach out at [email protected].

The description below focuses on Python examples, but Node and Java examples are also provided.

Install the SDK using the commands below:

pip install freeplay

npm install freeplay

<!-- Add the Freeplay SDK to your pom.xml -->
<dependency>
    <groupId>ai.freeplay</groupId>
    <artifactId>client</artifactId>
    <version>x.x.xx</version>
</dependency>

Set Secrets & Configuration

To make calls to Freeplay, you'll need to provide a Freeplay API key, a Freeplay Project ID, and at least one LLM Provider API key. Today we support all of the most common providers.

#Secrets
FREEPLAY_API_KEY='xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
OPENAI_API_KEY='sk-xxxxxxxxxxxxxx'
ANTHROPIC_API_KEY='sk-xxxxxxxxxxxxxx'

#Configuration
FREEPLAY_PROJECT_ID='xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
FREEPLAY_URL='https://acme.freeplay.ai/api'

You can find out how to access your Freeplay API key and Project ID in Account Configuration

Configure your Freeplay Client

All Freeplay functionality is exposed off of a top level Freeplay client.

from freeplay import Freeplay, RecordPayload, ResponseInfo, CallInfo
import os
from openai import OpenAI

# create your a freeplay client object
fpClient = Freeplay(
    freeplay_api_key=os.getenv("FREEPLAY_API_KEY"),
    api_base=freeplay_api_base,
)

import Freeplay from "freeplay/thin";

// create your freeplay client
const fpClient = new Freeplay({
    freeplayApiKey: process.env["FREEPLAY_KEY"],
    baseUrl: "https://acme.freeplay.ai/api",
});

import ai.freeplay.client.thin.Freeplay;

String projectId = System.getenv("FREEPLAY_PROJECT_ID");
String customerDomain = System.getenv("FREEPLAY_CUSTOMER_NAME");

Freeplay fpClient = new Freeplay(Config()
                                 .freeplayAPIKey(freeplayApiKey)
                                 .customerDomain(customerDomain)
                                );

// create the freeplay client
val fpClient = Freeplay(
  Freeplay.Config()
    .freeplayAPIKey(freeplayApiKey)
    .customerDomain(customerDomain)
)

Retrieve and format your Prompt

Freeplay manages and versions your prompts for you and allows you to retrieve prompts formatted for your specified LLM provider.

from freeplay import Freeplay, RecordPayload, ResponseInfo
import os
from openai import OpenAI
from vectorSearch import vector_search

# retreive and format your prompt
user_question = "What is Freeplay?"
prompt_vars = {"question": user_question, "supporting_information": vector_search(user_question)}
formatted_prompt = fpClient.prompts.get_formatted(project_id=os.getenv("FREEPLAY_PROJECT_ID"),
                                                	template_name="rag-qa",
                                                	environment="latest",
                                                	variables=prompt_vars)

// set the prompt variables
let userQuestion = "What is freeplay?"
let promptVars = {"question": userQuestion, "supporting_information": vectorSearch(userQuestion)};

// fetch a formatted prompt template
let formattedPrompt = await fpClient.prompts.getFormatted({
    projectId: projectID,
    templateName: "rag-qa",
    environment: "latest",
    variables: promptVars,
});

// set the prompt variables
val promptVars = mapOf("question" to "What is Freeplay?")

/* PROMPT FETCH */
val formattedPrompt = fpClient.prompts().getFormatted<String>(
  projectId,
  "rag-qa",
  "latest",
  promptVars
).await()

Make an LLM request using your chosen provider

Freeplay allows you to execute your own LLM calls for optimal flexibility and minimized latency.

from freeplay import Freeplay, RecordPayload, ResponseInfo
import os
import time
from openai import OpenAI

# make your llm call
start = time.time() # used to keep track of latency
openaiClient = OpenAI(api_key=openai_key)

chat_response = openaiClient.chat.completions.create(
    model=formatted_prompt.prompt_info.model, # model is specified in the freeplay prompt info
    messages=formatted_prompt.messages, # formatted messages based on your inserted variables and specified provider
    **formatted_prompt.prompt_info.model_parameters # model parameters assocated with your prompt (ie max_tokens, temperature)
)
end = time.time()

# add the LLM responses to your messages
all_messages = formatted_prompt.all_messages(
    {'role': chat_response.choices[0].message.role, 
     'content': chat_response.choices[0].message.content}
)

import OpenAI from "openai";

const openai = new OpenAI(process.env["OPENAI_API_KEY"]);

let start = new Date();
const chatCompletion = await openai.chat.completions.create({
    messages: formattedPrompt.messages,
    model: formattedPrompt.promptInfo.model,
});
let end = new Date();
console.log(chatCompletion.choices[0].message);

// update the messages
let messages = formattedPrompt.allMessages({
    role: chatCompletion.choices[0].message.role,
    content: chatCompletion.choices[0].message.content,
});

/* LLM CALL */
// set timer to measure latency
val startTime = System.currentTimeMillis()
val llmResponse = callOpenAI(
  objectMapper,
  openaiApiKey,
  formattedPrompt.promptInfo.model, // get the model name
  formattedPrompt.promptInfo.modelParameters, // get the model params
  formattedPrompt.getBoundMessages()
).await()
val endTime = System.currentTimeMillis()

// add the LLM response to the message set
val bodyNode = objectMapper.readTree(llmResponse.body())
val role = bodyNode.path("choices").path(0).path("message").path("role").asText()
val content = bodyNode.path("choices").path(0).path("message").path("content").asText()
println("Completion: " + content)

val allMessage: List<ChatMessage> = formattedPrompt.allMessages(
  ChatMessage(role, content)
)

Log your LLM call with Freeplay

Freeplay allows you to asynchronously log your LLM call to minimize customer latency and remove ourselves from your critical path.

# create a Freeplay session
session = fpClient.sessions.create()

# record your LLM call with Freeplay
payload = RecordPayload(
    all_messages=all_messages,
    inputs=prompt_vars,
    session_info=session, 
    prompt_info=formatted_prompt.prompt_info,
    call_info=CallInfo.from_prompt_info(formatted_prompt.prompt_info, start_time=start, end_time=end), 
    response_info=ResponseInfo(
        is_complete=chat_response.choices[0].finish_reason == 'stop'
    )
)
fpClient.recordings.create(payload)

// create a session
let session = fpClient.sessions.create({});

// record the LLM interaction with Freeplay
await fpClient.recordings.create({
    allMessages: messages,
    inputs: promptVars,
    sessionInfo: getSessionInfo(session),
    promptInfo: formattedPrompt.promptInfo,
    callInfo: getCallInfo(formattedPrompt.promptInfo, start, end),
    responseInfo: {
        isComplete: "stop_sequence" === chatCompletion.choices[0].stop
    }
});

/* RECORD */
// construct the call info from the prompt object
val callInfo = CallInfo.from(
  formattedPrompt.getPromptInfo(),
  startTime,
  endTime
)

// create the response info
val responseInfo = ResponseInfo("stop_sequence" == bodyNode.path("stop_reason").asText())

// create the session and get the session info
val sessionInfo = fpClient.sessions().create().sessionInfo

// build the final record payload
val recordResponse = fpClient.recordings().create(
  RecordInfo(
    allMessage,
    promptVars,
    sessionInfo,
    formattedPrompt.getPromptInfo(),
    callInfo,
    responseInfo
  )
).await()

This is just the start of what the Freeplay SDK enables. See the full docs here

View Sessions

Your LLM interactions will now be recorded to Freeplay and viewable from the Sessions tab. A Session is a flexible concept that is meant to be a logical grouping of Completions. In this case we will tie each Completion in a given user’s thread to a single Session.

In your Sessions view you can see a filterable table of all your Sessions as well as top level analytics for things like cost, latency and evaluation results.

You can click on each session to examine metadata like cost, token count, model and requests parameters, outputs and evaluations.

Quick Start

💡
Getting Freeplay set up for the first time

Before you get started

Create a new Project

Log into your Freeplay instance

Set up a Project

Create a Prompt Template

Craft your Prompt Template

Save and Deploy

Create Evaluation Criteria

Configuring Evaluation Criteria

Integrate Freeplay with your Application Code

Install the Freeplay SDK

Set Secrets & Configuration

Configure your Freeplay Client

Retrieve and format your Prompt

Make an LLM request using your chosen provider

Log your LLM call with Freeplay

View Sessions

💡Getting Freeplay set up for the first time

Before you get started

Create a new Project

Log into your Freeplay instance

Set up a Project

Create a Prompt Template

Craft your Prompt Template

Save and Deploy

Create Evaluation Criteria

Configuring Evaluation Criteria

Integrate Freeplay with your Application Code

Install the Freeplay SDK

Set Secrets & Configuration

Configure your Freeplay Client

Retrieve and format your Prompt

Make an LLM request using your chosen provider

Log your LLM call with Freeplay

View Sessions

💡
Getting Freeplay set up for the first time