IBM watsonx Orchestrate ADK tracing with Future AGI

pip install 'ibm-watsonx-orchestrate-sdk==2.18.2' langgraph langchain-openai

You do not need the ibm-watsonx-orchestrate CLI package for tracing. The collector runs from the otel/opentelemetry-collector-contrib image.

Save this as otelcol.yaml:

# OpenTelemetry Collector (contrib) between the watsonx Orchestrate Agentic SDK and Future AGI.
#
# Point the SDK's WXO_OTLP_ENDPOINT at this collector (http://<host>:4318/v1/traces). It then
#   1. names the Observe project (resource attribute project_name), which the SDK never sets,
#   2. maps the SDK's span.kind (llm, tool, agent, root, general) to Future AGI span kinds,
#   3. removes host and process details and stack traces, always,
#   4. removes prompts, outputs, tool arguments and results, error text and user ids,
#      unless KEEP_WXO_CONTENT=true,
#   5. forwards plain (uncompressed) OTLP/HTTP with X-Api-Key and X-Secret-Key.
# The Future AGI keys live only here. Headers that reach this collector (for example a
# watsonx Orchestrate bearer token from a Client-mode tracer) are never forwarded.
#
# Environment:
#   FI_TRACES_URL     where to forward, e.g. https://api.futureagi.com/tracer/v1/traces
#   FI_API_KEY        Future AGI API key
#   FI_SECRET_KEY     Future AGI secret key
#   FI_PROJECT_NAME   Observe project for spans that arrive without project_name
#   KEEP_WXO_CONTENT  "true" keeps prompts, outputs, tool data, error text and user ids (default: removed)

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318   # traces arrive on /v1/traces

processors:
  # `insert` keeps a project_name the app already set with TracerConfig(resource_attributes=...).
  resource/future_agi:
    attributes:
      - key: project_name
        value: ${env:FI_PROJECT_NAME}
        action: insert
      - key: project_type
        value: observe
        action: insert

  # The SDK writes its own span.kind attribute and llm.provider; Future AGI reads fi.span.kind
  # and gen_ai.provider.name. conversation.id becomes the Observe session. The SDK puts
  # user.id="Unknown User" on every span when no user is known; that placeholder is dropped.
  transform/map:
    error_mode: ignore
    trace_statements:
      - context: span
        statements:
          - set(attributes["fi.span.kind"], "LLM") where attributes["span.kind"] == "llm" and attributes["fi.span.kind"] == nil
          - set(attributes["fi.span.kind"], "TOOL") where attributes["span.kind"] == "tool" and attributes["fi.span.kind"] == nil
          - set(attributes["fi.span.kind"], "AGENT") where attributes["span.kind"] == "agent" and attributes["fi.span.kind"] == nil
          - set(attributes["fi.span.kind"], "CHAIN") where (attributes["span.kind"] == "root" or attributes["span.kind"] == "general") and attributes["fi.span.kind"] == nil
          - set(attributes["gen_ai.provider.name"], attributes["llm.provider"]) where attributes["llm.provider"] != nil and attributes["gen_ai.provider.name"] == nil
          - set(attributes["session.id"], attributes["conversation.id"]) where attributes["conversation.id"] != nil and attributes["session.id"] == nil
          - delete_key(attributes, "user.id") where attributes["user.id"] == "Unknown User"

  # Always removed. Stack traces carry local paths and user names; host and process
  # attributes appear when the app enables OpenTelemetry resource detectors.
  transform/always:
    error_mode: ignore
    trace_statements:
      - context: resource
        statements:
          - delete_matching_keys(attributes, "^(host|process)\\.")
      - context: spanevent
        statements:
          - delete_key(attributes, "exception.stacktrace")

  # Removed unless KEEP_WXO_CONTENT=true. The SDK records arguments and return values when a
  # decorator has capture_input/capture_output=True (IBM's examples set both), and error text
  # from the exception message.
  transform/content:
    error_mode: ignore
    trace_statements:
      - context: span
        conditions:
          - '"${env:KEEP_WXO_CONTENT:-false}" != "true"'
        statements:
          - delete_key(attributes, "input.value")
          - delete_key(attributes, "output.value")
          - delete_key(attributes, "tool.input")
          - delete_key(attributes, "tool.output")
          - delete_key(attributes, "agent.input")
          - delete_key(attributes, "agent.output")
          - delete_key(attributes, "user.id")
          - set(status.message, "error") where status.message != ""
      - context: spanevent
        conditions:
          - '"${env:KEEP_WXO_CONTENT:-false}" != "true"'
        statements:
          - delete_key(attributes, "exception.message")

exporters:
  otlp_http/future_agi:
    traces_endpoint: ${env:FI_TRACES_URL}
    encoding: proto
    compression: none          # optional: Future AGI cloud and self-hosted v1.56.0+ also accept gzip
    headers:
      X-Api-Key: ${env:FI_API_KEY}
      X-Secret-Key: ${env:FI_SECRET_KEY}

service:
  telemetry:
    metrics:
      level: none              # no self-metrics endpoint; this collector only carries traces
    logs:
      level: warn
  pipelines:
    traces:
      receivers: [otlp]
      processors: [resource/future_agi, transform/map, transform/always, transform/content]
      exporters: [otlp_http/future_agi]

Run it next to your agent, with your keys in the environment (-e FI_API_KEY copies the value from your shell without writing it on the command line):

export FI_API_KEY=... FI_SECRET_KEY=...
docker run -d --name wxo-otelcol -p 127.0.0.1:4318:4318 \
  -v "$PWD/otelcol.yaml:/etc/otelcol.yaml:ro" \
  -e FI_TRACES_URL=https://api.futureagi.com/tracer/v1/traces \
  -e FI_API_KEY -e FI_SECRET_KEY -e FI_PROJECT_NAME=weather-agent \
  otel/opentelemetry-collector-contrib:0.162.0 --config /etc/otelcol.yaml

The collector holds your Future AGI keys: anyone who can reach port 4318 can write to that project, so keep it on 127.0.0.1 or a private network.

export WXO_OTLP_ENDPOINT=http://localhost:4318/v1/traces

Use the full URL including /v1/traces: the SDK passes it to the OTLP exporter unchanged. Set it before your code creates the Tracer, which reads it once. Do not put Future AGI keys in WXO_OTLP_ENDPOINT or in OTEL_EXPORTER_OTLP_HEADERS for this process; the collector adds them.

Create one tracer per process and register it so the decorators use it:

from ibm_watsonx_orchestrate_sdk.observability import Tracer, TracerConfig, register_tracer
from ibm_watsonx_orchestrate_sdk.observability.attributes import ATTR_INPUT, ATTR_OUTPUT

# One tracer per process. Without a Client, TracerConfig reads the OTLP endpoint from
# WXO_OTLP_ENDPOINT and sends no auth header: the collector adds the Future AGI keys.
tracer = Tracer(TracerConfig(service_name="weather-agent"))
register_tracer(tracer)

Decorate tools as IBM documents, with @tool outermost:

@tool
@trace_tool_call(name="get_weather", tool_name="get_weather", capture_input=True, capture_output=True)
def get_weather(city: str) -> str:
    """Return the current weather for a city."""
    report = STATIONS.get(city.strip().lower())
    if report is None:
        raise ValueError(f"no weather station for {city}")
    return f"{city}: {report}"

@trace_llm_call records the model and provider but no token counts, so the model call uses the SDK’s context-manager form and record_completion:

def call_model(messages):
    # The LLM span uses the context-manager form: @trace_llm_call records no token counts.
    with tracer.start_llm_span("chat_model", model=MODEL, provider="openai") as span:
        span.set_attribute(ATTR_INPUT, json.dumps([m.model_dump(mode="json") for m in messages]))
        reply = llm.invoke(messages)
        usage = reply.usage_metadata or {}
        span.record_completion(
            prompt_tokens=usage.get("input_tokens"),
            completion_tokens=usage.get("output_tokens"),
            response_model=reply.response_metadata.get("model_name"),
        )
        span.set_attribute(ATTR_OUTPUT, reply.model_dump_json())
        return reply


@trace_agent_call(name="agent_node", agent_name="weather-agent", framework="langgraph",
                  capture_input=True, capture_output=True)
def agent_node(state: MessagesState) -> dict:
    return {"messages": [call_model(state["messages"])]}

Build the graph in a factory decorated with @configure_tracing, as IBM documents:

@configure_tracing
def create_agent(config: RunnableConfig):
    graph = StateGraph(MessagesState)
    graph.add_node("agent", agent_node)
    graph.add_node("tools", ToolNode([get_weather], handle_tool_errors=True))  # a failed tool call goes back to the model
    graph.add_edge(START, "agent")
    graph.add_conditional_edges("agent", route, ["tools", END])
    graph.add_edge("tools", "agent")
    return graph.compile()

Outside watsonx Orchestrate there is no platform trace to join, so each decorated call would start its own trace. Wrap each request in a root span, and shut the tracer down before the process exits so the last batch is sent:

    # A root span per request keeps the agent, model and tool spans of one run in one trace.
    with tracer.start_root_span("weather-agent", attributes={"conversation.id": conversation_id or str(uuid.uuid4())}) as root:
        root.set_attribute(ATTR_INPUT, question)
        result = agent.invoke(
            {"messages": [SystemMessage("Use get_weather for weather questions. Answer in one sentence."),
                          HumanMessage(question)]},
            config,
        )
        answer = result["messages"][-1].content
        root.set_attribute(ATTR_OUTPUT, answer)
    return answer

        tracer.shutdown()  # flushes the batch processor before the process exits

Pass a stable conversation_id (for example your chat thread id) to group a conversation’s traces into one Observe session.

export OPENAI_API_KEY=...
python agent.py "What is the weather in Tokyo?"

The trace appears in the Observe project named by FI_PROJECT_NAME within a few seconds.