Send Kong AI Gateway traces to Observe

About

Kong Gateway sits in front of your model providers. Its ai-proxy plugin handles the LLM request, and its bundled opentelemetry plugin can export what the gateway saw as a trace. Point that exporter at Future AGI and every request that goes through the gateway shows up in an Observe project with the model, token counts and latency, even when the calling app has no Future AGI code in it.

You configure Kong only. There is no Future AGI package, no custom Kong plugin and no Future AGI setting screen.

This is not client tracing. If you only installed traceAI-openai in an app that calls a Kong route, that traces the app’s call to Kong. This page traces the gateway itself, including callers you do not control. You can run both, but then the same call appears twice (once from the app, once from the gateway). They are not merged unless the app forwards a W3C traceparent (see Join a caller’s trace).

Before you start

• Kong Gateway Enterprise kong/kong-gateway:3.14.0.16 or 3.16.0.1. The open-source Kong 3.9 image does not emit the Gen AI spans used by this guide. Check your Kong agreement and licence requirements.
• A Future AGI API key and secret key, and the Observe project name you want the traces in.
• Kong must be started with tracing switched on. This is a process setting, not plugin config. Without it the plugin exports nothing:

KONG_TRACING_INSTRUMENTATIONS=request     # or all
KONG_TRACING_SAMPLING_RATE=1.0            # Kong's default samples almost nothing; also settable as config.sampling_rate on the plugin

Setup

Create kong.yml:

_format_version: "3.0"

services:
  - name: llm
    url: http://localhost:32000          # placeholder: ai-proxy picks the real upstream
    routes:
      - name: chat
        protocols: [http, https]
        paths: [/chat]
    plugins:
      - name: ai-proxy
        config:
          route_type: llm/v1/chat
          auth:
            header_name: Authorization
            header_value: "Bearer YOUR_PROVIDER_KEY"
          model:
            provider: openai
            name: gpt-4o-mini
          logging:
            log_statistics: true         # model + token counts on the span
            log_payloads: false          # true puts prompts and completions on the span

plugins:
  - name: opentelemetry
    config:
      traces_endpoint: https://api.futureagi.com/tracer/v1/traces   # self-hosted: http://<host>:4318/v1/traces
      headers:
        X-Api-Key: "YOUR_FI_API_KEY"
        X-Secret-Key: "YOUR_FI_SECRET_KEY"
      resource_attributes:
        project_name: my-kong-project
        project_type: observe
      sampling_rate: 1
      propagation:
        extract: [w3c]
        inject: [w3c]
        default_format: w3c

Field	Why
traces_endpoint	Kong’s plugin sends OTLP over HTTP with protobuf encoding, which is what the Future AGI collector accepts. gRPC is not used.
headers	X-Api-Key and X-Secret-Key. A bearer token does not work (the collector answers 401). Do not copy the X-Auth-Token name from Kong’s own examples, and do not use any Langfuse header.
resource_attributes.project_name	Required. Without it the collector rejects the whole batch with HTTP 400 (... have no project_name). project_type: observe makes it an Observe project.
sampling_rate	1 records every request. Lower it only on purpose.
propagation	W3C traceparent in and out, so the gateway span joins an existing trace.
ai-proxy logging.log_statistics	Needed for model and token counts on the Gen AI span.

Leave metrics, logs_endpoint and access_logs unset. The Future AGI collector takes traces only, so Kong would be pushing to routes that do not exist.

An example with ${...} placeholders is available in kong.yml.

Keep the keys out of the file

traces_endpoint and every value under headers can be a Kong vault reference. For example, enable KONG_VAULTS=env, provide the keys in the environment, and use X-Api-Key: "{vault://env/fi-api-key}" and X-Secret-Key: "{vault://env/fi-secret-key}" in the config. Quote the value in YAML, or the braces are read as a mapping. Without a vault reference the secret key sits in the plugin config and is visible to anyone who can read the Admin API.

Run it

docker run -d --name kong --memory 768m -p 8000:8000 \
  -e KONG_DATABASE=off -e KONG_DECLARATIVE_CONFIG=/kong/kong.yml \
  -e KONG_TRACING_INSTRUMENTATIONS=request -e KONG_TRACING_SAMPLING_RATE=1.0 \
  -v "$PWD/kong.yml:/kong/kong.yml:ro" kong/kong-gateway:3.16.0.1

curl -s localhost:8000/chat -H 'content-type: application/json' \
  -d '{"messages":[{"role":"user","content":"Say hi in 3 words."}]}'

Wait a few seconds after start-up before the first request. Spans are batched, so allow up to about ten seconds for the trace to appear in the project.

What you see in Observe

Each request is one trace with two spans:

Span	Observe type	What it carries
kong (the gateway request)	unknown	http.method, http.url, http.route, http.host, http.status_code, kong.request.id, client address. Duration is the gateway’s total time.
chat <model> (child)	LLM	gen_ai.operation.name, gen_ai.provider.name, gen_ai.request.model, gen_ai.response.model, gen_ai.response.id, gen_ai.response.finish_reasons, input and output token counts (when log_statistics is on).

Streaming requests can carry token counts when the caller asks the provider for usage with "stream_options": {"include_usage": true}. A tool-calling request appears as a chat span; tool names and arguments are not captured with payload logging off.

Not captured: cost (Observe computes it only from model and tokens, and shows nothing when either is missing), session id and user id (Kong does not set them; this page does not claim a header-to-attribute mapping), cache hits (Kong’s proxy-cache-advanced plugin answers before the tracing phase, so cache-HIT responses are not traced), metrics, and the provider key.

Privacy

Payload logging is Kong’s setting, and it is off in the config above.

With log_payloads: false, prompts, completions, tool arguments and results, provider keys and request headers are not exported.

With log_payloads: true Kong adds gen_ai.input.messages and gen_ai.output.messages. Kong warns they may contain prompts, model output, personal data, secrets and credentials. Turn this on only if you accept that. These two attributes are stored on the span, but they are not copied into the span’s Input and Output fields (those stay empty). Expect to find them in the span’s raw attributes.

Errors

Kong does not mark a failed request as an error span. An unreachable upstream can return 502 with a kong span whose status is UNSET and http.status_code is 502, and a chat span without tokens. The failure is visible in the HTTP status attribute and in the missing tokens, not as a red ERROR span, and no error text or stack trace was exported. Filter on http.status_code if you want failed gateway calls.

If Future AGI rejects the export (wrong keys, for example), the client’s request still succeeds. Kong logs response error: 401 from the exporter and retries, and the trace is lost.

Join a caller’s trace

If the caller sends a W3C traceparent, Kong uses its trace id and makes the caller’s span the parent of the kong span. If there is no traceparent, the kong span is the root of a new trace, and no Future AGI code is needed in the caller. propagation.default_format: w3c and extract: [w3c] are what make this work.

Limits

• This guide covers DB-less Kong AI Proxy chat requests with the openai provider and the Enterprise image versions listed above. It does not cover Konnect-managed data planes, Kubernetes ingress, Kong with PostgreSQL, other providers, advanced proxy, embeddings, file or batch routes.
• Kong controls sampling and the export queue. Future AGI cannot recover spans that Kong drops or fails to export.
• Create the Observe project before sending traces.
• Payloads use Kong’s parts message shape and appear in raw attributes rather than Input and Output fields.

Troubleshooting

What you see	What it means
Nothing arrives, no error in Kong logs	KONG_TRACING_INSTRUMENTATIONS is not set on the Kong process, or the sampling rate is too low. Both are process or plugin settings, not route settings.
Kong error log: response error: 401	The header names are not exactly X-Api-Key and X-Secret-Key, or the keys are wrong. A bearer token is rejected. Your proxied requests are unaffected.
Kong error log: response error: 400 ... have no project_name	resource_attributes.project_name is missing from the plugin config.
The kong span is there but no chat span	The request did not go through ai-proxy, the Kong build has no Gen AI spans (open-source 3.9 or older), or the response was a cache hit.
Tokens are missing	logging.log_statistics is off, the provider reported no usage, or a streaming call did not ask for usage. Missing is unknown, not zero.
Prompts are missing	log_payloads is false. That is the safe default. If it is true look in the span’s raw attributes, not the Input field.
First request right after start returns an error	Kong answers the Admin API before the proxy is ready. Wait until GET /status/ready on the Status API (status_listen) returns 200; it returns 503 until the config is loaded and the router is built.
Traces from Kong and from my app look duplicated	Both are exporting. Forward traceparent from the app to join them, or turn one off.

Next Steps