Ollama Integration with Future AGI for Local LLM Tracing

1. Installation

Install the instrumentation and OpenAI client packages.

pip install traceAI-openai openai

2. Set Environment Variables

Set up your environment variables to authenticate with FutureAGI.

import os

os.environ["FI_API_KEY"] = "your-futureagi-api-key"
os.environ["FI_SECRET_KEY"] = "your-futureagi-secret-key"

3. Initialize Trace Provider

Set up the trace provider to create a new project in FutureAGI, establish telemetry data pipelines.

from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType

trace_provider = register(
    project_type=ProjectType.OBSERVE,
    project_name="OLLAMA 3.2",
)

4. Instrument your Project

Use the OpenAI instrumentor for calls made through Ollama’s OpenAI-compatible client. If you use a different client, use its instrumentor instead.

from traceai_openai import OpenAIInstrumentor

OpenAIInstrumentor().instrument(tracer_provider=trace_provider)

5. Interact with Ollama

Interact with the Ollama as you normally would. Our Instrumentor will automatically trace and send the telemetry data to our platform. Make sure that Ollama is running and accessible from your project, and download the model before running the example:

ollama pull llama3.2:1b

from openai import OpenAI

client = OpenAI(
    base_url = 'http://localhost:11434/v1',
    api_key='ollama',  # Required by the client; ignored by the local Ollama server
)

response = client.chat.completions.create(
    model="llama3.2:1b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is OpenAI?"},
        ]
    )

print(response.choices[0].message.content)

Open the OLLAMA 3.2 project in Future AGI Observe to inspect the trace.