​
1. Installation

Install the FutureAGI package to access the observability framework.

pip install futureagi

​
2. Environment Configuration

Set up your environment variables to authenticate with FutureAGI services. These credentials enable:

  • Authentication with FutureAGI’s observability platform
  • Encrypted telemetry data transmission
import os
os.environ["FI_API_KEY"] = "your-futureagi-api-key"
os.environ["FI_SECRET_KEY"] = "your-futureagi-secret-key"

​
3. Configure Evaluation Tags

Define evaluation criteria for monitoring LLM responses. Evaluation tags allow you to:

  • Define custom evaluation criteria
  • Set up automated response quality checks
  • Track model performance metrics
from fi.integrations.otel.types import EvalName, EvalSpanKind, EvalTag, EvalTagType

eval_tags = [
    EvalTag(
        eval_name=EvalName.DETERMINISTIC_EVALS,
        value=EvalSpanKind.TOOL,
        type=EvalTagType.OBSERVATION_SPAN,
        config={
            "multi_choice": False,
            "choices": ["Yes", "No"],
            "rule_prompt": "Evaluate if the response is correct",
        },
        custom_eval_name="det_eval_litellm_1"
    )
]

​
4. Initialize Trace Provider

Set up the trace provider to establish the observability pipeline. The trace provider:

  • Creates a new project in FutureAGI
  • Establishes telemetry data pipelines
  • Configures version tracking
  • Sets up evaluation frameworks
from fi.integrations.otel import register
from fi.integrations.otel.types import ProjectType

trace_provider = register(
    project_type=ProjectType.EXPERIMENT,
    project_name="litellm_app",
    project_version_name="v1",
    eval_tags=eval_tags
)

​
5. Configure LiteLLM Instrumentation

Initialize the LiteLLM instrumentor to enable automatic tracing.

from fi.integrations.otel import LiteLLMInstrumentor

LiteLLMInstrumentor().instrument(tracer_provider=trace_provider)

​
6. Install Required Dependencies

Install the necessary LiteLLM components required for your project.

pip install litellm

​
7. Create LiteLLM Components

Set up your LiteLLM components with built-in observability.

import asyncio
import litellm

async def run_examples():
    # Simple single message completion call
    litellm.completion(
        model="gpt-3.5-turbo",
        messages=[{"content": "What's the capital of China?", "role": "user"}],
    )

    # Multiple message conversation completion call with added param
    litellm.completion(
        model="gpt-3.5-turbo",
        messages=[
            {"content": "Hello, I want to bake a cake", "role": "user"},
            {
                "content": "Hello, I can pull up some recipes for cakes.",
                "role": "assistant",
            },
            {"content": "No actually I want to make a pie", "role": "user"},
        ],
        temperature=0.7,
    )

    # Multiple message conversation acompletion call with added params
    await litellm.acompletion(
        model="gpt-3.5-turbo",
        messages=[
            {"content": "Hello, I want to bake a cake", "role": "user"},
            {
                "content": "Hello, I can pull up some recipes for cakes.",
                "role": "assistant",
            },
            {"content": "No actually I want to make a pie", "role": "user"},
        ],
        temperature=0.7,
        max_tokens=20,
    )

    # Completion with retries
    litellm.completion_with_retries(
        model="gpt-3.5-turbo",
        messages=[{"content": "What's the highest grossing film ever", "role": "user"}],
    )

    # Embedding call
    litellm.embedding(
        model="text-embedding-ada-002", input=["good morning from litellm"]
    )

    # Asynchronous embedding call
    await litellm.aembedding(
        model="text-embedding-ada-002", input=["good morning from litellm"]
    )

    # Image generation call
    litellm.image_generation(model="dall-e-2", prompt="cute baby otter")

    # Asynchronous image generation call
    await litellm.aimage_generation(model="dall-e-2", prompt="cute baby otter")

asyncio.run(run_examples())
InstructorLlama Index