Key Concepts

Projects

Projects are your primary workspaces within Freeplay. Each project should correspond to a distinct feature in your application that utilizes a language model. Invite team members to collaborate and structure your projects around specific prompts or a related set of prompts for effective testing and comparison.

Prompt Templates

These are blueprints for the prompts sent to a language model, crafted to elicit specific responses. They contain one or more Input Variables. that are populated via your application code at runtime based on user interactions.

Input Variable

This is a placeholder within a Prompt Template, represented using Mustache notation: {{variable_name}}. Input Variables are populated via your application code at runtime based on user interactions.

Sessions

A Session is a construct that represents a single, complete interaction with your language model-powered feature. This could be a single completion, a series of linked completions, or an entire conversation. Sessions are a flexible container that allows you to group Completions in whatever way makes sense for you application.

Completions

These are the individual responses generated by the language model for a single prompt within a Session. A Session can contain multiple completions, especially in scenarios like prompt chains or chatbot dialogs.

Evaluations

A user-designed label that you use to score all Sessions for a given Prompt Template that is specific to your domain.

Environments

These are designations for different versions of a Project, identifiable in your code. For instance, you might have a staging or prod environment. Environments enable you to continue using one prompt version in a specific setting while experimenting with another.

Datasets

Think of these as sets of labeled Sessions. They're used to assess new iterations of Prompt Templates or to conduct automated testing. Datasets include input variables and responses but don't contain the Prompt Templates themselves. They can be created from existing Sessions or by uploading example via JSONL.

Dataset Example

This represents a single entry in a Dataset, aimed at testing a particular variation of an updated Prompt Template or exporting for other purposes (like fine-tuning).

Test Run

A Test Run occurs when you execute a Test List in your code. It yields results for each test case using the current Project version. These runs can be benchmarked against established, labeled data from your Test List for comparison and analysis.