Tools

Tools allows LLMs to call external services. A tool schema describes the tool's capabilities and parameters. When invoked, the LLM provider responds with a tool call with the specified parameters from the schema. You can learn more about OpenAI tools here and Anthropic tools here.

How does Freeplay help with tools?

Freeplay supports the complete lifecycle of working with tools - from managing tool schemas and recording tool calls to surfacing detailed tool call information in observability and testing. This comprehensive approach enables rapid iteration and testing of your tools.

With the Freeplay web app, you can define tool schemas alongside your prompt templates. The Freeplay SDK formats the tool schema based on your LLM provider. The SDK also supports recording tool calls, associated schemas, and responses from tool calls. You have complete control over how much of this functionality you want to use.

Managing your tool schema with Freeplay

Freeplay enables you to define tools in a normalized format alongside your prompt template. Freeplay will then handle the translation of your tool definitions across providers so you can smoothly navigate across different providers. Simply provide a Name, Description and a JSON Schema to represent the parameters.

For example here is how you would define the parameters for a tool that fetches the weather

{
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "The city and state e.g. San Francisco, CA"
      },
      "unit": {
        "type": "string",
        "enum": [
          "c",
          "f"
        ]
      }
    },
    "additionalProperties": false,
    "required": [
      "location",
      "unit"
    ]
  }

properties represent the parameters of the tool and will be reflected in the arguments of the resulting tool call.

Each parameter has a type and either a description or an enum representing the possible options.

If your tool does not require any parameters to be passed you will want to set an empty schema like this

{
  "type": "object",
  "properties": {}
}

Here is how you add a Tool to a Prompt Template

• When adding or editing a prompt template with a supported LLM provider, you will see an "Add tool" button.

  

• Enter a name, description, and parameters for your tool

  

• Click on "Add tool" to add the tool to the prompt template. The prompt template will be in draft mode for you to run interactively in the editor. From there you can click save and create a new prompt template version with your tool schema attached.

Using the tool schema and recording tool calls

You can use the Freeplay SDK to fetch the tool schema as part of prompt retrieval and automatically format it for your LLM provider. We have recipes for OpenAI and Anthropic.

Using tools in a test run

Check out this recipe that demonstrates how to use tools programmatically in a test run.

Observability for tools

You can view recorded messages with tool calls and their schemas in individual Session details within the Observability tab. For test runs, all session details are available within individual test run details. Here's an example: