LLM Provider Java Integrations with Future AGI Tracing

Prerequisites

Complete the Java SDK setup first. All providers below need traceai-java-core and TraceAI.init() called before use.

Control captured content

The examples use default content capture, which can send prompts, responses, messages, raw request/response data and exception details to Future AGI. To hide standard input, output and message fields, replace TraceAI.initFromEnvironment() with:

import ai.traceai.TraceAI;
import ai.traceai.TraceConfig;

TraceAI.init(TraceConfig.builder()
    .hideInputs(true)
    .hideOutputs(true)
    .hideInputMessages(true)
    .hideOutputMessages(true)
    .build());

These are separate Java configuration controls. They do not suppress exception messages or stack traces recorded by setError, and are not a filter for every provider-specific attribute. Use non-sensitive sample data if your data policy does not permit those fields.

Google GenAI

Wraps the com.google.genai.Client for Google’s Gemini API.

<dependency>
    <groupId>com.github.future-agi.traceAI</groupId>
    <artifactId>traceai-java-google-genai</artifactId>
    <version>main-SNAPSHOT</version>
</dependency>

implementation 'com.github.future-agi.traceAI:traceai-java-google-genai:main-SNAPSHOT'

import ai.traceai.TraceAI;
import ai.traceai.googlegenai.TracedGenerativeModel;
import com.google.genai.Client;

TraceAI.initFromEnvironment();

Client client = Client.builder()
    .apiKey(System.getenv("GOOGLE_API_KEY"))
    .build();

// Note: model name is a constructor parameter
TracedGenerativeModel model = new TracedGenerativeModel(client, "gemini-2.0-flash");

// Simple generation
var response = model.generateContent("What is the capital of France?");
System.out.println(response.text());

// Multi-turn chat
var chat = model.startChat();
var reply = chat.sendMessage("Hello!");
System.out.println(reply.text());

// Token counting
var tokenCount = model.countTokens("How many tokens is this?");

Spans created:

• generateContent() - “Google GenAI Generate Content” (LLM)
• chat.sendMessage() - “Google GenAI Chat Message” (LLM)
• countTokens() - “Google GenAI Count Tokens” (LLM)

Open your project in Future AGI to inspect the spans.

Vertex AI

Google removed com.google.cloud.vertexai.generativeai from Vertex AI SDK releases after June 24, 2026. For new applications, follow Google’s migration guide to the Google Gen AI SDK. The legacy traceai-java-vertexai wrapper targets that removed package; do not apply it to a com.google.genai.Client.

The Google GenAI example above uses a Gemini API key. It is not a Vertex AI project/location authentication example.

Azure OpenAI

Wraps com.azure.ai.openai.OpenAIClient from the Azure SDK.

<dependency>
    <groupId>com.github.future-agi.traceAI</groupId>
    <artifactId>traceai-java-azure-openai</artifactId>
    <version>main-SNAPSHOT</version>
</dependency>

implementation 'com.github.future-agi.traceAI:traceai-java-azure-openai:main-SNAPSHOT'

import ai.traceai.TraceAI;
import ai.traceai.azure.openai.TracedAzureOpenAIClient;
import com.azure.ai.openai.OpenAIClient;
import com.azure.ai.openai.OpenAIClientBuilder;
import com.azure.core.credential.AzureKeyCredential;
import com.azure.ai.openai.models.ChatCompletionsOptions;
import com.azure.ai.openai.models.ChatRequestUserMessage;
import com.azure.ai.openai.models.EmbeddingsOptions;
import java.util.List;

TraceAI.initFromEnvironment();

OpenAIClient client = new OpenAIClientBuilder()
    .endpoint(System.getenv("AZURE_OPENAI_ENDPOINT"))
    .credential(new AzureKeyCredential(System.getenv("AZURE_OPENAI_API_KEY")))
    .buildClient();

TracedAzureOpenAIClient traced = new TracedAzureOpenAIClient(client);

// Chat completions - first arg is deployment name
var chatOptions = new ChatCompletionsOptions(List.of(
    new ChatRequestUserMessage("What is the capital of France?")
));
var response = traced.getChatCompletions("gpt-4o-mini", chatOptions);
System.out.println(response.getChoices().get(0).getMessage().getContent());

// Embeddings
var embeddingOptions = new EmbeddingsOptions(List.of("Hello world"));
var embeddings = traced.getEmbeddings("text-embedding-3-small", embeddingOptions);

Spans created:

• getChatCompletions() - “Azure OpenAI Chat Completion” (LLM)
• getEmbeddings() - “Azure OpenAI Embedding” (EMBEDDING)
• getCompletions() - “Azure OpenAI Completion” (LLM, legacy API)

Azure OpenAI captures tool call attributes when the model invokes tools, and handles all message types (System, User, Assistant, Tool, Function).

Ollama

Wraps io.github.ollama4j.OllamaAPI for local Ollama models.

<dependency>
    <groupId>com.github.future-agi.traceAI</groupId>
    <artifactId>traceai-java-ollama</artifactId>
    <version>main-SNAPSHOT</version>
</dependency>

implementation 'com.github.future-agi.traceAI:traceai-java-ollama:main-SNAPSHOT'

import ai.traceai.TraceAI;
import ai.traceai.ollama.TracedOllamaAPI;
import io.github.ollama4j.OllamaAPI;
import io.github.ollama4j.models.chat.OllamaChatMessage;
import io.github.ollama4j.models.chat.OllamaChatMessageRole;
import java.util.List;

TraceAI.initFromEnvironment();

OllamaAPI api = new OllamaAPI("http://localhost:11434");
TracedOllamaAPI traced = new TracedOllamaAPI(api);

// Generate
var result = traced.generate("llama3", "What is the capital of France?");
System.out.println(result.getResponse());

// Chat
var chatResult = traced.chat("llama3", List.of(
    new OllamaChatMessage(OllamaChatMessageRole.USER, "Hello!")
));

// Embeddings
var embedding = traced.embed("llama3", "Hello world");

// List models
var models = traced.listModels();

Spans created:

• generate() - “Ollama Generate” (LLM)
• chat() - “Ollama Chat” (LLM)
• embed() - “Ollama Embed” (EMBEDDING)
• listModels() - “Ollama List Models” (LLM)

Ollama spans include ollama.response_time_ms from the Ollama server’s own timing.

IBM Watsonx

Wraps the Watsonx Java SDK using reflection (like Anthropic). Compatibility depends on the methods exposed by the SDK version you use.

<dependency>
    <groupId>com.github.future-agi.traceAI</groupId>
    <artifactId>traceai-java-watsonx</artifactId>
    <version>main-SNAPSHOT</version>
</dependency>

implementation 'com.github.future-agi.traceAI:traceai-java-watsonx:main-SNAPSHOT'

TracedWatsonxAI is a wrapper reference, not a standalone Watsonx application. Pass an initialized SDK client to its constructor, then pass that SDK’s request objects to generateText, chat or embedText. See the wrapper source for the reflected method signatures; request construction depends on your Watsonx SDK.

Spans created:

• generateText() - “Watsonx Text Generation” (LLM)
• chat() - “Watsonx Chat” (LLM)
• embedText() - “Watsonx Embed” (EMBEDDING)

Watsonx spans include watsonx.project_id, watsonx.space_id, and watsonx.stop_reason.

Like Anthropic, the reflection approach means the client and request objects are typed as Object. Cast the return values to your SDK’s response types.

Common span attributes

Depending on the provider and operation, spans can include these attributes: