When it comes to Observability in Freeplay, there are three related objects to understand. From low to high level abstractions, there are:

• Completions: Atomic LLM calls made up of a prompt and a response or output from a model.
• Traces: Optionally used to group related completions, e.g. when multiple completions are used to generate chat turn or single agent flow
• Sessions: The container for all completions and traces that make up a customer interaction. These can be 1:1 with completions for a simple feature that just uses one prompt, or they can be very large at times e.g. an entire conversation thread between a single user and a chatbot over multiple hours.

Completions

The core, atomic unit for observability in Freeplay is a completion. A completion represents a single call to an LLM provider.

In Freeplay each completion is tied to a single prompt template version.

Most LLM applications involve a collection of completions to deliver a single outcome or user experience. This is where Sessions and Traces come into play: They provide a way to group completions in whatever way makes sense for your application logic.

Traces

Traces are the next level in the hierarchy. They can be optionally used to structure and visualize the logical behavior of your application within a Session. We do not recommend using traces for simply applications, e.g. if your user experience relies on running a single prompt one time.

Traces are helpful when you need a more fine-grained way to group completions within a Session. A trace can contain one or more completions, and a session can contain one or more traces.

In some applications, multiple LLM calls (completions) are made to compute a single output for the user. Consider an example in which a system takes in a user question, generates a query to retrieve data (Completion A), and then passes that data into another LLM call to generate the answer (Completion B). Traces allow you to bundle together completions A and B to express the relationship between them.

For more detail on implementation of Traces see our SDK Docs and a Full Code Recipe.

** While all Completions must be tied to a Session, Traces are entirely optional. You do not need to use them if they don't make sense in the context of your application logic.

Note that there is a special case for logging traces as part of multi-turn chatbots that lets you pass an input_question and output_answer to render the start and end of a chat turn between a user and a bot. Traces must be used to take advantage of this feature.

Sessions

Sessions are the highest level organizing principle in Freeplay for Observability. A session can contain one or more completions that are logically related to each other. We create a session every time you write logs to Freeplay, even when there's only one completion.

The simplest example of grouping completions into a session might be multiple user questions in the context of a single chatbot conversation. Each question results in a completion — a single call to an LLM provider to answer the question. Each of those completions can be logged together in a session to denote their relationship to each other (i.e all being part of the same conversation thread).

For more detail on recording completions, traces or sessions see our SDK Docs.