Flue tracing with Future AGI

Trace a Flue agent’s runs, model calls and tools in Future AGI with Flue’s OpenTelemetry adapter and an OpenTelemetry SDK.

This Node.js guide uses Flue 2.3.0 and Node 22.19 or later. It uses @opentelemetry/sdk-node 0.223.0, @opentelemetry/exporter-trace-otlp-proto 0.223.0 and @opentelemetry/sdk-trace-base 2.12.0. Get your Future AGI API and secret keys and choose an Observe project before you begin. See Setup Observability for keys and projects.

1. Install

In your Flue project:

npm install @flue/runtime@2.3.0 @flue/opentelemetry@2.3.0 @opentelemetry/api@1.9.1 @opentelemetry/sdk-node@0.223.0 @opentelemetry/sdk-trace-base@2.12.0 @opentelemetry/resources@2.12.0 @opentelemetry/exporter-trace-otlp-proto@0.223.0

@flue/opentelemetry declares @flue/runtime and @opentelemetry/api as peer dependencies, so keep the Flue packages on matching versions. The OTLP/HTTP protobuf exporter sends uncompressed requests, which is what the Future AGI collector accepts.

2. Set environment variables

export FI_API_KEY="your-futureagi-api-key"
export FI_SECRET_KEY="your-futureagi-secret-key"
export FI_PROJECT_NAME="flue-weather-agent"     # the Observe project the traces land in
# Self-hosted Future AGI only. Traces go to $FI_BASE_URL/tracer/v1/traces.
# export FI_BASE_URL="http://localhost:4318"    # default: https://api.futureagi.com
# export FI_CAPTURE_CONTENT=1                   # opt in to prompts, outputs and tool payloads

Set FI_BASE_URL to the origin only. The code adds /tracer/v1/traces. Flue loads a project .env file for flue run and vite dev, and the built server reads only the real environment.

3. Add the two files

The first file holds the Future AGI settings as plain functions of the environment: the endpoint, both auth headers, the resource attributes and the content policy.

// Future AGI settings for the Flue OpenTelemetry recipe. Pure functions of the
// environment, so they can be unit-tested without Flue or the network.
import type { ContentOption } from '@flue/opentelemetry';

type Env = Readonly<Record<string, string | undefined>>;

/** True when the app opted in to exporting prompts, outputs and tool payloads. */
export function captureContentEnabled(env: Env = process.env): boolean {
  return /^(1|true|yes|on)$/i.test((env.FI_CAPTURE_CONTENT ?? '').trim());
}

/**
 * The `content` option for createOpenTelemetryInstrumentation().
 *
 * Flue exports content by default: model messages, system instructions, tool
 * definitions, tool arguments and results, exception messages and stack traces.
 * `false` removes all of it. This recipe passes `false` unless the app sets
 * FI_CAPTURE_CONTENT=1. The opt-in keeps Flue's content (up to its 56 KiB
 * per-span budget) but still drops stack traces, which carry absolute local
 * paths.
 */
export function contentOption(env: Env = process.env): ContentOption {
  if (!captureContentEnabled(env)) return false;
  return {
    transform: (content, scope) => (scope.contentType === 'exception_stacktrace' ? undefined : content),
  };
}

/** The OTLP/HTTP traces URL: FI_BASE_URL (default: hosted) + /tracer/v1/traces. */
export function tracesUrl(env: Env = process.env): string {
  const base = (env.FI_BASE_URL || 'https://api.futureagi.com').replace(/\/+$/, '');
  return `${base}/tracer/v1/traces`;
}

/** Both auth headers are required; one alone is a failed setup. */
export function exporterHeaders(env: Env = process.env): Record<string, string> | null {
  const apiKey = env.FI_API_KEY?.trim();
  const secretKey = env.FI_SECRET_KEY?.trim();
  if (!apiKey || !secretKey) return null;
  return { 'X-Api-Key': apiKey, 'X-Secret-Key': secretKey };
}

/**
 * Resource attributes. Future AGI files spans by the RESOURCE attribute
 * `project_name`; a span attribute with that key does not count.
 */
export function resourceAttributes(env: Env = process.env): Record<string, string> {
  const project = env.FI_PROJECT_NAME?.trim() || 'flue-agent';
  return {
    project_name: project,
    project_type: 'observe',
    'service.name': env.OTEL_SERVICE_NAME?.trim() || project,
  };
}

The second file starts the SDK, registers Flue’s instrumentation and arranges the flush.

// Sends Flue's OpenTelemetry spans to Future AGI. Import this module once, before
// any agent runs: from src/app.ts for a server, and from the agent module for
// `flue run` (which never loads app.ts).
import { createOpenTelemetryInstrumentation } from '@flue/opentelemetry';
import { instrument } from '@flue/runtime';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto';
import { resourceFromAttributes } from '@opentelemetry/resources';
import { NodeSDK } from '@opentelemetry/sdk-node';
import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-base';

import { contentOption, exporterHeaders, resourceAttributes, tracesUrl } from './futureagi.ts';

export function installFutureAGITracing(env: NodeJS.ProcessEnv = process.env): (() => Promise<void>) | undefined {
  const headers = exporterHeaders(env);
  if (!headers) {
    console.warn('[futureagi] FI_API_KEY and FI_SECRET_KEY must both be set; Flue tracing is off.');
    return undefined;
  }

  // 1. The application owns the SDK: Flue's package never creates one.
  //    Batches spans and exports every 5 seconds (OTEL_BSP_* env vars tune it).
  const processor = new BatchSpanProcessor(new OTLPTraceExporter({ url: tracesUrl(env), headers })); // uncompressed OTLP/HTTP protobuf
  const sdk = new NodeSDK({
    resource: resourceFromAttributes(resourceAttributes(env)),
    // The default detectors add host.name, host.id, process.owner (the OS user
    // name), process.command_args and process.executable.path (absolute local
    // paths) to every export. Turn them off.
    autoDetectResources: false,
    spanProcessors: [processor],
    // Traces only. By default NodeSDK also starts OTLP metric and log exporters
    // that post to http://localhost:4318 (or OTEL_EXPORTER_OTLP_ENDPOINT). The
    // Future AGI collector takes traces only, and the failed exports would
    // delay shutdown by about ten seconds.
    metricReaders: [],
    logRecordProcessors: [],
  });
  sdk.start();

  // 2. Register Flue's instrumentation. Traces only: do not pass `meter` or
  //    `logger`, because the Future AGI collector accepts traces only.
  const instrumentation = createOpenTelemetryInstrumentation({ content: contentOption(env) });

  // 3. Flush.
  //    - `flue run` disposes module-scope instrumentation before it exits.
  //    - A standalone script that boots the runtime itself exits when its event
  //      loop drains: flush on `beforeExit`.
  //    - A built server (`node dist/server.mjs`) does not dispose instrumentation
  //      created in app.ts and calls process.exit() as soon as it has stopped, so
  //      an async flush on SIGTERM loses the race. Export each submission's spans
  //      as soon as it settles instead.
  const dispose = instrument({
    ...instrumentation,
    observe(event, ctx) {
      const result = instrumentation.observe(event, ctx);
      if (event.type === 'submission_settled') void processor.forceFlush().catch(() => {});
      return result;
    },
    async dispose() {
      instrumentation.dispose(); // ends spans that are still open
      await sdk.shutdown(); // flushes the batch processor
    },
  });
  process.once('beforeExit', () => void dispose());
  return dispose;
}

/** Call to flush and stop tracing yourself, for example at the end of a script. */
export const shutdownTracing: () => Promise<void> = installFutureAGITracing() ?? (async () => {});

Why the code looks like this:

• project_name is a resource attribute. Future AGI files spans by the resource attribute project_name. A span attribute with that key does not count, and a batch without it is rejected. service.name is not read as a project name.
• autoDetectResources: false. Disable automatic resource detection to avoid adding host and process metadata, such as host names, OS user names and absolute executable paths, to exported spans.
• metricReaders: [] and logRecordProcessors: []. NodeSDK otherwise starts OTLP metric and log exporters that post to http://localhost:4318. Flue also emits metrics and logs, but only into a meter or logger it is given. The Future AGI collector takes traces only, so the recipe passes neither.
• content. false unless FI_CAPTURE_CONTENT=1. See Privacy.
• Both auth headers are required. The keys travel only as X-Api-Key and X-Secret-Key headers to the collector, never on a span and never to the model provider. If either key is missing, the module prints one warning and leaves tracing off; the agent still runs.
• Three flush paths. See How it behaves.

4. Import it before any agent runs

Flue registers instrumentation at module scope. Import telemetry.ts first in src/app.ts, and also from the agent module if you run it with flue run, because flue run loads the agent module and never app.ts:

import './telemetry.ts'; // first import: the SDK and Flue's instrumentation are installed before any agent runs
import { createAgentRouter } from '@flue/runtime/routing';
import { Hono } from 'hono';

import { Weather } from './agents/weather.ts';

const app = new Hono();
app.route('/agents/weather', createAgentRouter(Weather));

export default app;

'use agent';
import { useModel, useTool } from '@flue/runtime';

import '../telemetry.ts'; // `flue run` loads this module, not app.ts

Importing it twice is safe: the module runs once. Calling instrument(createOpenTelemetryInstrumentation()) a second time is not. Flue throws InstrumentationAlreadyInstalledError, so keep one registration.

5. Run and check

Run your agent and open its trace in your Observe project. The following commands use the example application:

# One-shot, as a CLI or CI step.
npx flue run src/agents/weather.ts --message 'Use get_weather for the city "Paris".'

# A server.
npx vite build && node dist/server.mjs
curl -X POST http://localhost:3000/agents/weather/demo-1 -H 'content-type: application/json' \
  -d '{"kind":"user","body":"Use get_weather for the city \"Paris\"."}'

# A standalone script (start() and init(), no server and no app.ts).
node scripts/ask.ts 'Use get_weather for the city "Paris".'

invoke_agent Weather          one per run; status ERROR if the run fails
├── chat gpt-6-luna           model, provider, input and output tokens, finish reason
├── execute_tool get_weather  tool name, call id, ERROR if the tool throws
└── chat gpt-6-luna

All three children hang directly off the invoke_agent span; Flue does not nest the tool span under the model call. The root is named after the agent function. Each run is its own trace.

Flue span	Future AGI type	What you see
invoke_agent <agent>	unknown (see Limitations)	Duration, status. No token counts.
chat <model>	llm	Model, provider (the id you registered), input and output tokens, finish reason, reasoning level
execute_tool <tool>	tool	Tool name, call id, duration, error.type and ERROR status when the tool throws

A failed model call, for example a wrong model API key, shows ERROR on the chat span and on the invoke_agent span. A tool that throws shows ERROR on its own span only: the model sees the error and the run can still finish, so the root stays OK.

What is captured

Data	Default	FI_CAPTURE_CONTENT=1
Span names, timing, parent links, status code	Yes	Yes
Model id, provider name, token usage (input, output, cache read, cache creation), finish reason, reasoning level, stream flag	Yes	Yes
Tool name, tool call id, tool type	Yes	Yes
error.type, and an exception event that carries only the exception type	Yes	Yes
Flue ids and names: flue.instance.id, flue.submission.id, flue.operation.id, flue.turn.id, flue.agent.name, flue.harness.name, flue.session.name, gen_ai.conversation.id	Yes	Yes
Model endpoint host and port (server.address, server.port)	Yes	Yes
Prompts and replies (gen_ai.input.messages, gen_ai.output.messages), system instructions, tool definitions and descriptions	No	Yes, up to 56 KiB per span
Tool arguments and results (gen_ai.tool.call.arguments, gen_ai.tool.call.result)	No	Yes
Error text (the span status message and exception.message)	No	Yes
Stack traces (exception.stacktrace)	No	No
Resource attributes	project_name, project_type, service.name only	The same
Flue metrics and logs	Never	Never

flue.instance.id is the id in your agent’s route (the demo-1 in /agents/weather/demo-1, or --id for flue run). If you build that id from an email address or a user name, it is exported even with content off. Use opaque ids.

Privacy

Default: metadata only. The recipe passes content: false to createOpenTelemetryInstrumentation(). Flue then removes model messages, system instructions, tool definitions, tool arguments and results, and exception messages and stack traces from every span.

Opt in: FI_CAPTURE_CONTENT=1. The recipe then passes a content policy, { transform }, instead of false. Flue’s content is exported, truncated to its 56 KiB per-span budget, and the policy drops exception_stacktrace content in both modes. Error messages and tool arguments are content: if your tools put local paths or secrets in them, the opt-in exports them. Redact in the same transform before you opt in; Flue’s OpenTelemetry guide documents its content.transform(content, scope) hook and its scope.contentType values.

How it behaves

• Flushing. The batch processor exports every 5 seconds. The recipe flushes when a submission settles, disposes the instrumentation when flue run exits, and flushes a standalone script on beforeExit. The Flue 2.3.0 built server calls process.exit() during shutdown, so an asynchronous flush started on SIGTERM may not finish. Runs settling during server shutdown, crashes and forced termination can still lose spans.
• Exporter down. The workflow can finish even when the collector is unreachable. Fix the endpoint or adjust OTEL_EXPORTER_OTLP_TRACES_TIMEOUT if the exporter delays exit.
• Retried model calls. A failed model call and its retry appear as separate chat spans.
• Sessions. Each run is its own trace. Flue sets gen_ai.conversation.id for a persisted conversation but no session.id, so Future AGI does not group runs into sessions. Filter on gen_ai.conversation.id to see one conversation.
• Streaming. One chat span per model call, ended when the stream ends.

Limitations

• The root can show type “unknown”. If your collector does not map gen_ai.operation.name=invoke_agent to an agent type, the root still has its name, timing and status. Model and tool spans use the chat and execute_tool operations.
• Content display depends on your collector version. With FI_CAPTURE_CONTENT=1, model messages are exported as gen_ai.input.messages and gen_ai.output.messages. Collectors that support these keys fill the Input and Output panes; on older versions, inspect them in the span’s Attributes panel.
• Tokens are per model call. The invoke_agent span carries none, so summing over a trace counts each call once.
• dispatch() does not propagate trace context, according to Flue’s OpenTelemetry guide. Runs started by dispatch() become their own traces.
• Flue does not emit spans for embeddings, retrieval, memory operations or evaluations, per its guide.

Older guides

Older guides, written for Flue’s beta releases, use createOpenTelemetryObserver({ exportContent }) with observe(...). That API is gone: Flue’s 2.x migration guide says it became createOpenTelemetryInstrumentation() registered with instrument(...), and the exportContent option became the content option. The old function is not exported by @flue/opentelemetry 2.2.2 or 2.3.0, and neither is createOpenTelemetryObservability. Mind the default: the current package exports content unless you pass content: false.

Troubleshooting

• No traces in the project. Check that FI_API_KEY and FI_SECRET_KEY are both set (the module warns and turns tracing off otherwise), that the resource carries project_name, and that FI_BASE_URL is an origin without a path.
• No traces from flue run. Import telemetry.ts from the agent module. flue run never loads src/app.ts.
• The last run of a script or server is missing. In a standalone script, call await shutdownTracing() after the agent finishes. For a server, let active runs settle and exports finish before stopping it. SIGTERM or SIGINT alone does not guarantee the final export: the Flue 2.3.0 built server can exit before an asynchronous shutdown flush finishes. SIGKILL and crashes lose queued spans.
• InstrumentationAlreadyInstalledError. Something already registered Flue’s OpenTelemetry instrumentation. Keep one instrument(createOpenTelemetryInstrumentation()) call.
• flue run exits ten seconds late. The exporter cannot reach FI_BASE_URL. Fix the URL, or set OTEL_EXPORTER_OTLP_TRACES_TIMEOUT to a few thousand milliseconds.
• Prompts or tool arguments appear on spans. FI_CAPTURE_CONTENT is set, or content was left out of createOpenTelemetryInstrumentation(), which means content on.
• host.name or process.owner on the resource. The NodeSDK was created without autoDetectResources: false.
• flue run fails with Failed to load tsconfig 'astro/tsconfigs/strict'. The Flue project sits inside another project whose tsconfig.json extends an Astro preset. Add a tsconfig.json to the Flue project folder.