Skip to main content

​
Overview

Cost estimates in Freeplay give you a clear view of how you are spending your API tokens. Freeplay breaks down costs by project, prompt template, and evaluation so you can trace your spend and identify where costs are highest. Freeplay also provides rough estimates of evaluation costs before you run them, helping you make informed decisions about your testing and evaluation strategy.

​
Where to find cost estimates

You can view cost information in two places within Freeplay.

​
Account settings —> Usage

The Usage tab in Account Settings is the primary source of truth for costs and cost estimates. Navigate to Account Settings and select the Usage tab. This page provides two key metrics:
  • App spend - estimated spend based on logged token counts to Freeplay. This reflects the cost of running your application’s LLM calls, not money spent in or through Freeplay itself.
  • Spend via Freeplay - the total spend from activity within the Freeplay platform. This includes:
    • Online evaluations and auto-categories
    • Test runs
    • Prompt playground usage
    • AI features

​
Estimated evaluation costs

On the evaluations page, each evaluation displays an estimated cost. This estimate gives you a rough idea of how much it costs to run that evaluation over your completions. The estimate is calculated as follows:
  1. Takes the average input and output tokens for the specific prompt template or agent (including input variables) from the last week
  2. Calculates the average cost based on those token counts
This is a rough estimate and is subject to change based on volume, sampling rate, and other factors.

​
How costs are calculated

Freeplay bases all cost estimates on token usage. Token costs are calculated using up-to-date pricing information from each provider. Cost estimates may vary if you:
  • Use different LLM providers
  • Have custom deployment configurations (e.g., Azure OpenAI, AWS Bedrock)
  • Have negotiated pricing agreements with providers

​
Custom per-token costs

For certain providers such as LiteLLM, you can provide custom input and output per-token cost information. This allows Freeplay to reflect your actual costs more accurately when you have special pricing or use self-hosted models. To configure custom token costs, see LiteLLM Proxy.

​
Frequently asked questions

  • Model selection — switching to a smaller or less expensive model for tasks that do not require the most capable model
  • Prompt optimization — reducing token counts by writing more concise prompts
  • Sampling rate — adjusting the sampling rate for online evaluations to run them on a subset of completions rather than all of them
  • Evaluation frequency — running test evaluations less frequently or on smaller datasets during development
Freeplay uses publicly available pricing from each LLM provider to estimate the cost per input and output token. These rates are updated regularly to reflect the latest published pricing.
If you use self-hosted models, custom deployments, or have negotiated pricing with providers, the default per-token costs may not reflect your actual spend. For providers that support it (such as LiteLLM), you can configure custom per-token costs in Freeplay to get more accurate estimates. For other providers, treat the estimates as a relative benchmark for comparing costs across prompts and models.