Prompts

Retrieve your prompt templates from the Freeplay server. All methods associated with your Freeplay prompt template are accessible from the client.prompts namespace.

Methods Overview

Some SDKs will use camel case rather than snake case depending on convention for the given language

Method Name	Parameters	Description
`get_formatted`	`project_id:` string `template_name:` string `environment:` string	Get a formatted prompt template object with variables inserted, messages formatted for the configured LLM provider (e.g. OpenAI, Anthropic), and model parameters for the LLM call.
`get`	`project_id:` string `template_name:` string `environment:` string	Get a prompt template by environment.Note: The prompt template will not have variables substituted or be formatted for the configured LLM provider.

Get a Formatted Prompt

Get a formatted prompt object with variables inserted, messages formatted for the associated LLM provider, and model parameters to use for the LLM call. This is the most convenient method for most prompt fetch use cases given that formatting is handled for you server side in Freeplay.

The environment parameter determines which version of your prompt is fetched. Default environment names are:

"prod" - Production environment
"dev" - Development environment
"latest" - Latest version (use for development/testing)

Learn more about managing prompts across environments and multi-environment configuration patterns.

# get a formatted prompt
formatted_prompt = fp_client.prompts.get_formatted(
  project_id=project_id,
  template_name="template_name",
  environment="latest",
  variables={"keyA": "valueA"}
)

# Sample use in an LLM call
start = time.time()
chat_response = openai_client.chat.completions.create(
  model=formatted_prompt.prompt_info.model,
  messages=formatted_prompt.llm_prompt,
  **formatted_prompt.prompt_info.model_parameters
)
end = time.time()

# add the response to your message set
all_messages = formatted_prompt.all_messages({
  'role': chat_response.choices[0].message.role,
  'content': chat_response.choices[0].message.content
})

// set the prompt variables
let promptVars = {"keyA": "valueA"};

// fetch a formatted prompt template
let formattedPrompt = await fpClient.prompts.getFormatted({
    projectId: projectID,
    templateName: "template_name",
    environment: "latest",
    variables: promptVars,
});

import ai.freeplay.client.thin.resources.prompts.FormattedPrompt;

// create the prompt variables
Map<String, Object> promptVars = Map.of("keyA", "valueA");
// get a formatted prompt
CompletableFuture<FormattedPrompt<Object>> formattedPrompt = fpClient.prompts()
  .getFormatted(projectId, "template_name", "latest", promptVars, null);

// set the prompt variables
val promptVars = mapOf("keyA" to "valA")

/* PROMPT FETCH */
val formattedPrompt = fpClient.prompts().getFormatted<String>(
  projectId,
  "template-name",
  "latest",
  promptVars
).await()

Get a Prompt Template

Get a prompt template object that does not yet have variables inserted and has messages formatted in consistent LLM provider agnostic structure. It is particularly useful when you want to reuse a prompt template with different variables in the same execution path, like Test Runs. This method gives you more control to handle formatting in your own code rather than server side in Freeplay, but requires a few more lines of code.

# get an unformatted prompt template
template_prompt = fp_client.prompts.get(
  project_id=project_id,
  template_name="template_name",
  environment="latest"
)
# to format the prompt
formatted_prompt = template_prompt.bind({"keyA": "valueA"}).format()

// get a prompt template
let promptTemplate = await fpClient.prompts.get({
  projectId: projectID,
  templateName: "album_bot",
  environment: "latest",
});

// format the prompt template
// set the prompt variables
let promptVars = { keyA: "valueA" };
// format the prompt template
let formattedPrompt = promptTemplate.bind(promptVars).format();

import ai.freeplay.client.thin.resources.prompts.TemplatePrompt;
import ai.freeplay.client.thin.resources.prompts.FormattedPrompt;

// create the prompt variables
Map<String, Object> promptVars = Map.of("keyA", "valueA");

// get prompt template
CompletableFuture<TemplatePrompt> promptTemplate = fpClient.prompts().get(
  projectId, "template_name", "environment");

// bind and format client side
 FormattedPrompt<Object> formattedPrompt = promptTemplate.join().bind(promptVars).format(
   promptTemplate.get().getPromptInfo().getFlavorName()
 );

// set the prompt variables
val promptVars = mapOf("keyA" to "valA")

// fetch prompt template
val promptTemplate = fpClient.prompts().get(
  projectId,
  "template-name",
  "latest"
).await()

// bind and format client side
val formattedPrompt = promptTemplate.bind(promptVars).format<String>()

Using History with Prompt Templates

history is a special object in Freeplay prompt templates for managing state over multiple LLM interactions by passing in previous messages. It accepts an array of prior messages when relevant. Before using history in the SDK, you must configure it on your prompt template. See more details in the Multi-Turn Chat Support section. Once you have history configured for a prompt template, you can pass it during the formatting process. The history messages will be inserted wherever you have your history placeholder in your prompt template.

previous_messages = [{"role": "user", "content": "what are some dinner ideas..."},
                      {"role": "assistant", "content": "here are some dinner ideas..."}]
prompt_vars = {"question": "how do I make them healthier?"}

formatted_prompt = fp_client.prompts.get_formatted(
  project_id=project_id,
  template_name="SamplePrompt",
  environment="latest",
  variables=prompt_vars,
  history=previous_messages  # pass the history messages here
)

# llm_prompt contains messages formatted for the provider

print(formatted_prompt.llm_prompt)

# output:

[
{'role': 'system', 'content': 'You are a polite assistant...'},
{'role': 'user', 'content': 'what are some dinner ideas...'},
{'role': 'assistant', 'content': 'here are some dinner ideas...'},
{'role': 'user', 'content': 'how do I make them healthier?'}
]

See a full implementation of using history in the context of a multi-turn chatbot application here

Using Tool Schemas with Prompt Templates

You can define tool schemas alongside your prompt templates. The Freeplay SDK will format the tool schema based on the configured LLM provider, so you can pass the tool schema to the LLM provider as is.

# get a formatted prompt
formatted_prompt = fp_client.prompts.get_formatted(
    project_id=project_id,
    template_name="template_name",
    environment="latest",
    variables={"keyA": "valueA"}
)

# Sample use in an LLM call
start = time.time()
chat_response = openai_client.chat.completions.create(
    model=formatted_prompt.prompt_info.model,
    messages=formatted_prompt.llm_prompt,
    # Pass the tool schema to the LLM call
    tool_schema=formatted_prompt.tool_schema,
    **formatted_prompt.prompt_info.model_parameters
)
end = time.time()

// set the prompt variables
let promptVars = { keyA: "valueA" };

// fetch a formatted prompt template
let formattedPrompt = await fpClient.prompts.getFormatted({
  projectId: projectID,
  templateName: "template_name",
  environment: "latest",
  variables: promptVars,
});

// Sample use in an LLM call
const chatCompletion = await openai.chat.completions.create({
  messages: formattedPrompt.llmPrompt,
  model: formattedPrompt.promptInfo.model,
  toolSchema: formattedPrompt.toolSchema,
  ...formattedPrompt.promptInfo.modelParameters,
});

val variables = mapOf("keyA" to "valA")
fpClient.prompts()
    .getFormatted<List<ChatMessage>>(
        projectId,
        "my-prompt",
        "latest",
        variables,
    ).thenCompose { formattedPrompt ->
        val startTime = System.currentTimeMillis()
        callAnthropic(
            objectMapper,
            anthropicApiKey,
            formattedPrompt.promptInfo.model,
            formattedPrompt.promptInfo.modelParameters,
            formattedPrompt.formattedPrompt,
            formattedPrompt.systemContent.orElse(null),
            formattedPrompt.toolSchema
        ).thenApply { Triple(formattedPrompt, it, startTime) }
    }
    .join()

Record Evals From Your Code

Freeplay allows you to record client-side executed evals to Freeplay during the record step (more here). Code evals are useful for running objective assertions or pairwise comparisons against ground truth data. They are passed as a key-value pair and can be associated with either a regular Session or a Test Run session.

# record the results
payload = RecordPayload(
  project_id=project_id,
  all_messages=all_messages,
  inputs=prompt_vars,
  session_info=session.session_info,
  prompt_version_info=prompt_info,
  call_info=call_info,
  eval_results={
      "valid schema": True,
      "valid category": True,
      "string distance": 0.81
  }
)
completion_info = fp_client.recordings.create(payload)

// record the interaction with Freeplay
const completionInfo = await fpClient.recordings.create({
  projectId,
  allMessages: messages,
  inputs: promptVars,
  sessionInfo: getSessionInfo(session),
  promptVersionInfo: formattedPrompt.promptInfo,
  callInfo: callInfoPayload,
  evalResults: {
    "valid schema": true,
    "valid category": true,
    "string distance": 0.81,
  },
});

import ai.freeplay.client.thin.Freeplay
import ai.freeplay.client.thin.resources.prompts.ChatMessage
import ai.freeplay.client.thin.resources.recordings.CallInfo
import ai.freeplay.client.thin.resources.recordings.RecordInfo

/* RECORD */
// get the existing parameters
val modelParams = formattedPrompt.promptInfo.modelParameters;
modelParams["presence_penalty"] = 0.8;
modelParams["n"] = 5
// construct the call info from the prompt object
val callInfo = CallInfo(
  formattedPrompt.promptInfo.provider,
  formattedPrompt.promptInfo.model,
  startTime,
  endTime,
  modelParams
)

// create the session and get the session info
val sessionInfo = fpClient.sessions().create().sessionInfo

val evalResults = mapOf(
    "valid schema" to true,
    "valid category" to true,
    "string distance" to 0.81
)

// build the final record payload
val recordResponse = fpClient.recordings().create(
  RecordInfo(
    projectId,
    allMessage,
    ).inputs(promptVars)
    .sessionInfo(sessionInfo)
    .promptVersionInfo(formattedPrompt.promptInfo)
    .callInfo(callInfo)
    .evalResults(evalResults)
  )

Using your own IDs

Freeplay allows you to provide your own client-side UUIDs for both Sessions and Completions. This can be useful if you already have natural identifiers in your application code. Providing your own Completion Id also allows you to store the completion Id to be used for recording customer feedback without having to wait for the record call to complete. Thus making the record call entirely non-blocking.

from freeplay import Freeplay, RecordPayload, CallInfo, SessionInfo
from uuid import uuid4

## PROMPT FETCH

# set the prompt variables
prompt_vars = {"keyA": "valueA"}

# get a formatted prompt
formatted_prompt = fp_client.prompts.get_formatted(
  project_id=project_id,
  template_name="template_name",
  environment="latest",
  variables=prompt_vars
)

## LLM CALL
# Make an LLM call to your provider of choice
start = time.time()
chat_response = openai_client.chat.completions.create(
  model=formatted_prompt.prompt_info.model,
  messages=formatted_prompt.llm_prompt,
  **formatted_prompt.prompt_info.model_parameters
)
end = time.time()

# add the response to your message set
all_messages = formatted_prompt.all_messages({
  'role': chat_response.choices[0].message.role,
  'content': chat_response.choices[0].message.content
})

## RECORD

### CUSTOM IDS

# create your Ids (must be UUIDs)

session_id = uuid4()
completion_id = uuid4()

# Create sessionInfo with custom Ids

session_info = SessionInfo(
session_id=session_id,
custom_metadata={'keyA': 'valueA'}
)

### Extra data

call_info = CallInfo(
provider=formatted_prompt.prompt_info.provider,
model=formatted_prompt.prompt_info.model,
start_time=s,
end_time=e,
model_parameters=all_params # pass the full parameter set
)

# build the record payload
payload = RecordPayload(
  project_id=project_id,
  all_messages=all_messages,
  inputs=prompt_vars,
  session_info=session_info,
  completion_id=completion_id,
  prompt_version_info=formatted_prompt.prompt_info,
  call_info=call_info,
)

# record the LLM interaction
fp_client.recordings.create(payload)

import Freeplay, { getCallInfo, getSessionInfo } from "freeplay";
import { v4 as uuidv4 } from "uuid";
import OpenAI from "openai";

/* PROMPT FETCH */
// set the prompt variables
let promptVars = {"pop_star": "Taylor Swift"};
// fetch a formatted prompt template
let formattedPrompt = await fpClient.prompts.getFormatted({
    projectId,
    templateName: "template_name",
    environment: "latest",
    variables: promptVars,
});

/* LLM CALL */
// make the llm call
const openai = new OpenAI(process.env["OPENAI_API_KEY"]);

let start = new Date();
const chatCompletion = await openai.chat.completions.create({
    messages: formattedPrompt.llmPrompt,
    model: formattedPrompt.promptInfo.model,
  	...formattedPrompt.promptInfo.modelParameters
});
let end = new Date();
console.log(chatCompletion.choices[0].message);

// update the messages
let messages = formattedPrompt.allMessages({
    role: chatCompletion.choices[0].message.role,
    content: chatCompletion.choices[0].message.content,
});

/* RECORD */
// create your own Ids
const sessionId = uuidv4();
const completionId = uuidv4();

// record the LLM interaction with Freeplay
await fpClient.recordings.create({
    projectId,
    allMessages: messages,
    inputs: promptVars,
    sessionInfo: {sessionId: sessionId, customMetadata: {"keyA": "valueA"}},
  	completionId: completionId,
    promptVersionInfo: formattedPrompt.promptInfo,
    callInfo: getCallInfo(formattedPrompt.promptInfo, start, end)
});

package ai.freeplay.example.kotlin

import ai.freeplay.client.thin.Freeplay
import ai.freeplay.client.thin.resources.prompts.ChatMessage
import ai.freeplay.client.thin.resources.recordings.CallInfo
import ai.freeplay.client.thin.resources.recordings.RecordInfo
import ai.freeplay.example.java.ThinExampleUtils.callOpenAI
import com.fasterxml.jackson.databind.ObjectMapper
import java.util.UUID
import kotlinx.coroutines.future.await
import kotlinx.coroutines.runBlocking

private val objectMapper = ObjectMapper()

fun main(): Unit = runBlocking {
    // set your environment variables
    val openaiApiKey = System.getenv("OPENAI_API_KEY")
    val freeplayApiKey = System.getenv("FREEPLAY_API_KEY")
    val projectId = System.getenv("FREEPLAY_PROJECT_ID")
    val customerDomain = System.getenv("FREEPLAY_CUSTOMER_NAME")

    // create your url from your customer domain
    val baseUrl = String.format("https://%s.freeplay.ai/api", customerDomain)

    // create the freeplay client
    val fpClient = Freeplay(
            Freeplay.Config()
                    .freeplayAPIKey(freeplayApiKey)
                    .customerDomain(customerDomain)
    )

    // set the prompt variables
    val promptVars = mapOf("keyA" to "valueA")

    /* PROMPT FETCH */
    val formattedPrompt = fpClient.prompts().getFormatted<String>(
            projectId,
            "template-name",
            "latest",
            promptVars
    ).await()

    /* LLM CALL */
    // set timer to measure latency
    val startTime = System.currentTimeMillis()
    val llmResponse = callOpenAI(
            objectMapper,
            openaiApiKey,
            formattedPrompt.promptInfo.model, // get the model name
            formattedPrompt.promptInfo.modelParameters, // get the model params
            formattedPrompt.getBoundMessages()
    ).await()
    val endTime = System.currentTimeMillis()

    // add the LLM response to the message set
    val bodyNode = objectMapper.readTree(llmResponse.body())
    val role = bodyNode.path("choices").path(0).path("message").path("role").asText()
    val content = bodyNode.path("choices").path(0).path("message").path("content").asText()
    println("Completion: " + content)

    val allMessage: List<ChatMessage> = formattedPrompt.allMessages(
            ChatMessage(role, content)
    )

    /* RECORD */
    // construct the call info from the prompt object
    val callInfo = CallInfo.from(
            formattedPrompt.getPromptInfo(),
            startTime,
            endTime
    )

    // create the session and get the session info
    val sessionInfo = fpClient.sessions().create().sessionInfo
  	// create your completion Id
  	val completionId = UUID.randomUUID()

  // build the final record payload
    val recordResponse = fpClient.recordings().create(
      RecordInfo(
          projectId,
          allMessage,
          )
          .completionId(completionId)
          .inputs(promptVars)
          .sessionInfo(sessionInfo)
          .promptVersionInfo(formattedPrompt.promptInfo)
          .callInfo(callInfo)
          .evalResults(evalResults)
        )
    ).await()
    println("Completion Record Succeeded with ${recordResponse.completionId}")

}

Getting Started

Recording Data

Testing

Reference

Prompts

Methods Overview

Get a Formatted Prompt

Get a Prompt Template

Using History with Prompt Templates

Using Tool Schemas with Prompt Templates

Record Evals From Your Code

Using your own IDs

​Methods Overview

​Get a Formatted Prompt

​Get a Prompt Template

​Using History with Prompt Templates

​Using Tool Schemas with Prompt Templates

​Record Evals From Your Code

​Using your own IDs

Methods Overview

Get a Formatted Prompt

Get a Prompt Template

Using History with Prompt Templates

Using Tool Schemas with Prompt Templates

Record Evals From Your Code

Using your own IDs