Overview
Building LLM applications with modern frameworks is great—until you need to understand what’s actually happening under the hood. Traditional observability tools weren’t built for the nuances of LLM interactions: token counts, model parameters, prompt templates, tool calls, and multi-step agent workflows.
That’s where OpenTelemetry (OTel) comes in. By integrating Freeplay with OTel, you get purpose-built LLM observability that works with any framework or orchestration approach. Whether you’re using Langgraph, building custom agents, or mixing frameworks, Freeplay captures what matters—automatically.
Freeplay uses OpenTelemetry as a protocol to record LLM observability data. It’s not intended to provide general application telemetry for non-LLM related code, so sending arbitrary telemetry to Freeplay will not work. Freeplay supports traces that conform to the OpenInference semantic conventions.
Why OTel + Freeplay?
- Framework flexibility: Your team uses Langgraph. Another team built custom agents. A third is evaluating Google ADK. With OTel, one integration supports all of them.
- LLM-native insights: We automatically capture model parameters, token counts, tool schemas, and prompt templates—the data you actually need to improve your AI applications.
- No architectural changes: Freeplay observes your orchestration logic without becoming part of it. Your code stays clean, your flexibility stays intact.
- Built on standards: OpenInference semantic conventions mean your instrumentation is portable and future-proof.
Freeplay focuses on LLM observabilityWe only record traces and spans containing meaningful LLM information per OpenInference semantic conventions—not general application telemetry.Recommended approach:
- Use OpenInference instrumentation libraries when available (easiest option)
- Follow OpenInference semantic conventions for custom instrumentation
- For advanced control, use our OTel-compliant API directly
If your framework lacks an OpenInference library, you can still record data using standard OpenInference attributes like input.value, output.value, and gen_ai.request.model. See the supported attributes reference below. Getting Started
Step 1: Install dependencies
pip install freeplay opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk import trace as trace_sdk
import os
freeplay_api_url = "https://app.freeplay.ai/api"
exporter = OTLPSpanExporter(
endpoint=f"{freeplay_api_url}/v0/otel/v1/traces",
headers={
"Authorization": f"Bearer {os.environ['FREEPLAY_API_KEY']}",
"X-Freeplay-Project-Id": os.environ["FREEPLAY_PROJECT_ID"],
},
)
tracer_provider = trace_sdk.TracerProvider()
tracer_provider.add_span_processor(SimpleSpanProcessor(exporter))
Getting your credentials: Step 3: Instrument your application
We recommend using OpenInference instrumentation libraries when available. For example, with Google ADK:
from google.adk.instrumentation import GoogleADKInstrumentor
GoogleADKInstrumentor().instrument(tracer_provider=tracer_provider)
Step 4: Enable debugging (optional but recommended)
During development, print spans to your console for easier debugging:
from opentelemetry.sdk.trace.export import ConsoleSpanExporter
tracer_provider.add_span_processor(
SimpleSpanProcessor(ConsoleSpanExporter())
)
Framework examples
Langgraph/Langchain
Complete end-to-end example integrating OTel with Langgraph
Custom instrumentation
If you’re building with a framework that doesn’t have an OpenInference instrumentation library, you can manually instrument your code following OpenInference conventions:
from opentelemetry import trace
tracer = trace.get_tracer(__name__)
with tracer.start_as_current_span("llm_call") as span:
# Set OpenInference attributes
span.set_attribute("openinference.span.kind", "LLM")
span.set_attribute("gen_ai.request.model", "gpt-4")
span.set_attribute("input.value", user_query)
# Make your LLM call
response = your_llm_call(user_query)
# Record output and token usage
span.set_attribute("output.value", response)
span.set_attribute("gen_ai.usage.input_tokens", token_count.input)
span.set_attribute("gen_ai.usage.output_tokens", token_count.output)
Your span must have a kind value of either SPAN_KIND_INTERNAL or SPAN_KIND_SERVER.Your spans must have an attribute named openinference.span.kind with a value of LLM, AGENT, CHAIN, or TOOL.Other spans will be ignored.Freeplay only records data data described below, not general application telemetry.
Key concepts
Span types
Freeplay processes different span types based on openinference.span.kind:
- LLM: Direct LLM API calls with prompts, completions, and token usage
- AGENT: Higher-level agentic workflows with decision-making
- CHAIN: Sequential operations or pipelines
- TOOL: External tool or function calls
Set the appropriate span kind to ensure Freeplay correctly interprets your traces.
Sessions and traces
Use session and trace identifiers to organize related interactions:
# Link multiple traces to the same conversation session
span.set_attribute("session.id", session_id)
# Or use Freeplay-specific attributes (takes precedence)
span.set_attribute("freeplay.session.id", session_id)
# Name your agent for better organization
span.set_attribute("agent.name", "customer_support_agent")
Environment attribute
You should add environment attribute to your spans that corresponds with the
prompt template environment that you are using. For example, if you are using a
prompt template in the “production” environment, you should set the environment
attribute to “production”.
span.set_attribute("freeplay.environment", "production")
# or "staging", "development", etc.
Supported attributes reference
OpenInference attributes
Freeplay maps OpenInference attributes to internal fields for consistent observability. These are the standard attributes you should use when instrumenting your LLM applications:
| OTEL Attribute | Freeplay Field | Payload Type | Notes |
|---|
openinference.span.kind | N/A | RecordPayload | Determines span type (LLM, AGENT, CHAIN, TOOL) |
gen_ai.request.model | completion.model_name | RecordPayload | LLM model name |
llm.model_name | completion.model_name | RecordPayload | Alternative LLM model name |
gen_ai.usage.input_tokens | completion.token_counts.prompt_token_count | RecordPayload | Input token count |
llm.token_count.prompt | completion.token_counts.prompt_token_count | RecordPayload | Alternative input token count |
gen_ai.usage.output_tokens | completion.token_counts.return_token_count | RecordPayload | Output token count |
llm.token_count.completion | completion.token_counts.return_token_count | RecordPayload | Alternative output token count |
llm.provider | completion.provider_name | RecordPayload | LLM provider (openai, anthropic, google) |
gen_ai.system | completion.provider_name | RecordPayload | Alternative provider identifier |
llm.input_messages.* | completion.api_messages | RecordPayload | Input messages (flattened array format) |
llm.output_messages.* | completion.api_messages | RecordPayload | Output messages (flattened array format) |
llm.tools.* | completion.tool_schema | RecordPayload | Tool/function definitions |
llm.prompt_template.variables | completion.inputs | RecordPayload | Variables applied to template (merged into inputs) |
llm.invocation_parameters | completion.llm_parameters | RecordPayload | Parameters used during LLM invocation (JSON) |
input.value | trace.input | TracePayload | Trace input data |
output.value | trace.output | TracePayload | Trace output data |
tool.name | trace.name | TracePayload | Tool span name |
agent.name | trace.agent_name | TracePayload | Agent span name |
session.id | completion.session_id | RecordPayload | Session identifier |
session.id | trace.session_id | TracePayload | Session identifier |
Freeplay-specific attributes
Enhance your traces with Freeplay-specific metadata for tighter integration with Freeplay features:
| OTEL Attribute | Freeplay Field | Payload Type | Notes |
|---|
freeplay.session.id | completion.session_id | RecordPayload | Freeplay session ID (takes precedence over session.id) |
freeplay.session.id | trace.session_id | TracePayload | Freeplay session ID (takes precedence over session.id) |
freeplay.environment | completion.tag | RecordPayload | Environment tag (e.g., “production”, “staging”) |
freeplay.prompt_template.id | completion.prompt_template_id | RecordPayload | Link to Freeplay prompt template |
freeplay.prompt_template.version.id | completion.prompt_template_version_id | RecordPayload | Specific prompt template version |
freeplay.test_run.id | completion.test_run_id | RecordPayload | Associate with test run |
freeplay.test_case.id | completion.test_case_id | RecordPayload | Associate with specific test case |
freeplay.input_variables | completion.inputs | RecordPayload | Input variables (JSON) |
freeplay.metadata.* | completion.custom_metadata | RecordPayload | Custom metadata (flattened dict format) |
freeplay.eval_results.* | completion.eval_results | RecordPayload | Evaluation results (flattened dict format) |
Next steps
Once you’ve integrated OTel with Freeplay, you can:
- View traces in the Freeplay observability dashboard
- Run evaluations on your logged interactions to measure quality
- Build datasets from production traces for systematic testing
- Monitor performance across different model versions and configurations
- Track costs with automatic token usage and cost calculations
Additional resources
