Azure API Management OTLP egress

About

Route application traces through Azure API Management Consumption tier to Future AGI using OTLP/HTTP. Your application creates spans; an APIM policy adds the Future AGI credentials and forwards them. APIM does not create model spans. Native AI Gateway telemetry is outside this recipe.

Use the OpenAI instrumentor to create client spans. The egress policy changes how those spans reach Future AGI, not what they contain.

Configure API Management as an OTLP egress

1. What Future AGI needs

Field	Value
URL	https://api.futureagi.com/tracer/v1/traces (hosted), or <your collector>/tracer/v1/traces when self-hosted
Protocol	OTLP/HTTP, protobuf. Not gRPC.
Headers	X-Api-Key and X-Secret-Key. Authorization: Bearer ... always gets 401.
Resource attribute	project_name, set by the caller. A batch without it is refused with 400.

Create the Observe project’s keys first. The caller names the project; APIM does not.

2. Configure APIM

1. Named values. Create two secret named values, fi-api-key and fi-secret-key. A Key Vault reference is the better home for them.
2. API. Create an API with the URL suffix tracer and one operation, POST /v1/traces. The Python SDK normalizes the exporter path to /tracer/v1/traces, replacing any path in FI_BASE_URL. Set FI_BASE_URL to the APIM host only.
3. Policy. Add this to the operation (or the API), and replace the base-url with your collector’s if you self-host:

<policies>
  <inbound>
    <base />
    <set-backend-service base-url="https://api.futureagi.com" />
    <rewrite-uri template="/tracer/v1/traces" copy-unmatched-params="false" />
    <set-header name="X-Api-Key" exists-action="override">
      <value>{{fi-api-key}}</value>
    </set-header>
    <set-header name="X-Secret-Key" exists-action="override">
      <value>{{fi-secret-key}}</value>
    </set-header>
    <!-- the caller's APIM subscription key and Authorization header are for APIM, not for Future AGI -->
    <set-header name="Ocp-Apim-Subscription-Key" exists-action="delete" />
    <set-header name="Authorization" exists-action="delete" />
  </inbound>
  <backend><base /></backend>
  <outbound><base /></outbound>
  <on-error><base /></on-error>
</policies>

The same policy is available in the example folder.

3. Point the app at APIM

Install the published client packages in Python 3.10–3.13:

pip install "traceAI-openai==0.1.10" "fi-instrumentation-otel==1.1.0" "openai==3.31.0"
export FI_BASE_URL="https://<your-apim>.azure-api.net"
export OTEL_EXPORTER_OTLP_COMPRESSION=none

Keep FI_API_KEY and FI_SECRET_KEY unset in the application environment; the SDK adds them to outgoing headers when both are present. Store the Future AGI keys in APIM instead.

import os
from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType
from traceai_openai import OpenAIInstrumentor

# FI_BASE_URL=https://<your-apim>.azure-api.net     (host only; no FI_API_KEY or FI_SECRET_KEY on this machine)
trace_provider = register(
    project_type=ProjectType.OBSERVE,
    project_name="my-chatbot",
    headers={"Ocp-Apim-Subscription-Key": os.environ["APIM_SUBSCRIPTION_KEY"]},  # APIM's key, not Future AGI's
)
OpenAIInstrumentor().instrument(tracer_provider=trace_provider)

Any other OpenTelemetry exporter works the same way: send OTLP/HTTP protobuf, uncompressed (OTEL_EXPORTER_OTLP_COMPRESSION=none), with project_name on the resource. If your APIM API requires a subscription key, send it in Ocp-Apim-Subscription-Key. Short scripts must flush before exit (trace_provider.force_flush()).

4. What shows up in Observe

Observe shows the spans emitted by the caller. For example, an instrumented agent can produce a trace like this:

azure-apim-demo        agent   your own parent span, session id on it
├── ChatCompletion     llm     model and tokens
├── ChatCompletion     llm     tool call requested by the model
├── tool lookup_record tool    your own span
├── ChatCompletion     llm     final answer
└── ChatCompletion     llm     failure case: status ERROR

Model, token and prompt data come from the caller. Session and user data appear only if the caller includes them; attributes removed by an APIM policy do not reach Observe.

Privacy for the egress recipe

The caller’s privacy settings apply. traceAI-openai captures prompts, responses and tool-call content by default. Hide input and output content in the app before instrumentation, not in APIM:

export FI_HIDE_INPUTS=true
export FI_HIDE_OUTPUTS=true
export FI_HIDE_LLM_INVOCATION_PARAMETERS=true

These settings are not a general-purpose payload scrubber. With the pinned instrumentation version, error text and tool definitions can still contain sensitive data; review the masking controls before exporting sensitive workloads.

APIM adds a second place text can leak: its own logging. If you enable request or response body logging, header logging, or a diagnostic that records request payloads for this API, APIM can store your prompts (and, if you log the outgoing request, the Future AGI keys). Keep payload and header logging off for the OTLP API. Future AGI cannot prevent this.

Scope

This guide covers APIM Consumption tier and uncompressed OTLP/HTTP protobuf traces. It does not cover other APIM tiers, request-size limits, native AI Gateway telemetry, gRPC, logs, or metrics.

Troubleshooting

Symptom	Check
App works but no trace arrives	FI_BASE_URL must be the APIM host with no path; API suffix tracer; subscription header; project_name resource; uncompressed OTLP; flush before exit.
APIM returns 401	The subscription key is missing or wrong.
Backend returns 401	The policy must set both secret named values and remove the caller’s Authorization header.
Backend returns 400	Check project_name on the resource.
Keys or prompts appear in APIM logs	Turn off header and payload logging for the OTLP API and rotate exposed keys.

Example and references

The example folder includes the policy and client app. app.py accepts FI_BASE_URL (your APIM host), APIM_SUBSCRIPTION_KEY, and LLM_BASE_URL, LLM_API_KEY, LLM_MODEL for your model endpoint. It deliberately rejects Future AGI keys in the app environment. Use --scenario chat for a single model call and flush.

• Microsoft set-header policy
• Microsoft set-backend-service policy
• Microsoft rewrite-uri policy
• Langfuse APIM egress guide