Trace LiteLLM calls with Future AGI

Trace Python LiteLLM SDK calls in Future AGI with traceAI-litellm.

Trace LiteLLM Python SDK calls in Observe. For a Proxy server, see LiteLLM Proxy tracing.

Install

Use Python 3.10 to 3.13. The traceAI-litellm package declares LiteLLM 1.43.0 or later as a dependency.

pip install 'traceAI-litellm==0.1.12' 'litellm>=1.43.0'

Warning

If you installed version 0.1.11 or earlier, upgrade with the command above. Those versions record the arguments you pass to LiteLLM’s completion calls, sync and async, in the span. That includes an api_key, auth headers or any other credential passed in the call. Image generation calls that pass the prompt positionally are recorded the same way. If you passed a credential in either kind of call, rotate it: upgrading does not change spans you already sent. Keys set only through environment variables were not recorded.

Set credentials and capture preferences

Set these in your shell, then run the Python blocks below in one script. Choose a model available to your OpenAI account; LiteLLM’s openai/ prefix selects that provider.

export FI_API_KEY="your-futureagi-api-key"
export FI_SECRET_KEY="your-futureagi-secret-key"
export OPENAI_API_KEY="your-openai-api-key"
export LITELLM_MODEL="openai/your-available-model"

Instrumentation can send messages, model responses and error details to Future AGI. To hide input and output values and messages, set FI_HIDE_INPUTS=true and FI_HIDE_OUTPUTS=true before initialization. See mask span attributes. These switches are not a general scrubber for custom attributes or exception text; avoid secrets in request data and error messages.

Register the trace provider

Name the Observe project that will receive spans.

from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType

trace_provider = register(
    project_type=ProjectType.OBSERVE,
    project_name="litellm_project",
)

Instrument LiteLLM

Initialize instrumentation before making requests.

from traceai_litellm import LiteLLMInstrumentor

LiteLLMInstrumentor().instrument(tracer_provider=trace_provider)

Run a request

import os
import litellm

try:
    response = litellm.completion(
        model=os.environ["LITELLM_MODEL"],
        messages=[{"role": "user", "content": "What's the capital of India?"}],
    )
    print(response.choices[0].message.content)
finally:
    trace_provider.shutdown()

shutdown() sends pending spans before this short-lived process exits.

View the trace

Open Observe in Future AGI and select litellm_project to inspect the request’s input, output, timing and available model usage. Usage fields depend on the response returned by the model provider.

Was this page helpful?

Questions & Discussion