​
1. Installation

First install the traceAI package to access the observability framework

pip install traceAI-anthropic

pip install anthropic

​
2. Environment Configuration

Set up your environment variables to authenticate with both FutureAGI and Anthropic services. These credentials enable:

  • Secure access to FutureAGI’s observability platform
  • Authentication with Anthropic’s API
  • Encrypted telemetry data transmission
import os
os.environ["FI_API_KEY"] = FI_API_KEY
os.environ["FI_SECRET_KEY"] = FI_SECRET_KEY
os.environ["ANTHROPIC_API_KEY"] = ANTHROPIC_API_KEY

​
3. Configure Evaluation Tags

Define evaluation criteria for monitoring LLM responses. Evaluation tags allow you to:

  • Define custom evaluation criteria
  • Set up automated response quality checks
  • Track model performance metrics

Click here here to learn how to configure eval tags for observability.

import anthropic
from traceai_anthropic import AnthropicInstrumentor
from fi_instrumentation import register
from fi_instrumentation.fi_types import (
    EvalName,
    EvalSpanKind,
    EvalTag,
    EvalTagType,
    ProjectType,
)

eval_tags = [
    EvalTag(
        eval_name=EvalName.DETERMINISTIC_EVALS,
        value=EvalSpanKind.TOOL,
        type=EvalTagType.OBSERVATION_SPAN,
        config={
            "multi_choice": False,
            "choices": ["Yes", "No"],
            "rule_prompt": "Evaluate if the response is correct",
        },
        custom_eval_name="det_eval_anthropic_1"
    )
]

​
4. Instrument the Anthropic Client

Instrument the Anthropic client to enable telemetry collection. This step ensures that all interactions with the Anthropic API are tracked and monitored.

trace_provider = register(
    project_type=ProjectType.EXPERIMENT,
    eval_tags=eval_tags,
    project_name="FUTURE_AGI432423",
    project_version_name="v1",
)

AnthropicInstrumentor().instrument(tracer_provider=trace_provider)

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-3-5-sonnet-20240620",
    max_tokens=1024,
    tools=[
        {
            "name": "get_weather",
            "description": "Get the current weather in a given location",
            "input_schema": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The unit of temperature, either 'celsius' or 'fahrenheit'",
                    },
                },
                "required": ["location"],
            },
        },
        {
            "name": "get_time",
            "description": "Get the current time in a given time zone",
            "input_schema": {
                "type": "object",
                "properties": {
                    "timezone": {
                        "type": "string",
                        "description": "The IANA time zone name, e.g. America/Los_Angeles",
                    }
                },
                "required": ["timezone"],
            },
        },
    ],
    messages=[
        {
            "role": "user",
            "content": "What is the weather like right now in New York?"
            " Also what time is it there? Use necessary tools simultaneously.",
        }
    ],
)
print(response)