Eve (Vercel) tracing with Future AGI

Trace an eve agent’s turns, model calls and tools in Future AGI using eve’s OpenTelemetry instrumentation.

This guide uses eve 0.80.0 and Node.js 24 or later. Get your Future AGI API and secret keys and choose an Observe project before you begin. For an eve agent, don’t add @traceai/vercel or a separate registerOTel call: eve owns the tracer provider. For AI SDK calls outside eve, see the Vercel AI SDK integration.

1. Install

In your eve project:

npm install @vercel/otel @opentelemetry/sdk-trace-base

@vercel/otel supplies the OTLP/HTTP exporter (eve’s own guide uses it). @opentelemetry/sdk-trace-base supplies the batching processor.

2. Set environment variables

export FI_API_KEY="your-futureagi-api-key"
export FI_SECRET_KEY="your-futureagi-secret-key"
export FI_PROJECT_NAME="eve-weather-agent"     # the Observe project the traces land in
# Self-hosted Future AGI only. Traces go to $FI_BASE_URL/tracer/v1/traces.
# export FI_BASE_URL="http://localhost:4318"   # default: https://api.futureagi.com
# export FI_CAPTURE_CONTENT=1                  # opt in to prompts, outputs and tool payloads

Set FI_BASE_URL to the origin only. The code adds /tracer/v1/traces.

3. Add the instrumentation files

eve loads every file in agent/instrumentation/ at startup. Each file there is one instrumentation slot, so helper code goes in agent/lib/.

The first file names the Future AGI project. Future AGI files spans by the resource attribute project_name; a span attribute with that key does not count. An agent can declare otel() only once, so if you already have agent/instrumentation/otel.ts, add the two keys to its resource instead.

// Process-wide OpenTelemetry settings. Declare otel() once per agent.
// Future AGI files every span under the resource attribute `project_name`.
import { otel } from "eve/instrumentation/otel";

export default otel({
  resource: {
    project_name: process.env.FI_PROJECT_NAME ?? "eve-agent",
    project_type: "observe",
  },
});

The second file is the Future AGI destination. eve combines it with any other destination you declare.

// The Future AGI destination. eve combines it with any other destination file.
import { otelIntegration } from "eve/instrumentation/otel";

import { futureAGIExportPolicy, futureAGISpanProcessor } from "../lib/futureagi.ts";

export default otelIntegration({
  // Metadata only unless FI_CAPTURE_CONTENT=1. Applies to this destination
  // only: eve's local traces and Workflow retention are unchanged.
  exportPolicy: futureAGIExportPolicy(),
  // Drops eve's durable-workflow spans, then batches and exports the rest.
  spanProcessors: [futureAGISpanProcessor()],
});

The third file holds the exporter, the scope filter and the export policy.

// Future AGI destination pieces for eve's OpenTelemetry pipeline.
//
// eve sends every span in the process to each destination, including the
// durable-workflow plumbing under instrumentation scope "workflow" (dozens of
// spans per turn, in separate traces). This file keeps eve's agent spans
// (scope "eve.agent") and anything else you add, and drops the plumbing.
import type { Context } from "@opentelemetry/api";
import {
  BatchSpanProcessor,
  type ReadableSpan,
  type Span,
  type SpanExporter,
  type SpanProcessor,
} from "@opentelemetry/sdk-trace-base";
import { OTLPHttpProtoTraceExporter } from "@vercel/otel";
import type { SpanExportPolicy } from "eve/instrumentation/otel";

/** Instrumentation scopes that never reach Future AGI. */
export const DROPPED_SCOPES: ReadonlySet<string> = new Set([
  "workflow", // durable orchestration: steps, queues, replays, routes
  "@vercel/otel/fetch", // outbound HTTP spans, when fetch instrumentation is on
]);

type Env = Readonly<Record<string, string | undefined>>;

/** The instrumentation scope name of a span (OTel JS SDK 2.x and 1.x shapes). */
export function scopeName(span: ReadableSpan | Span): string {
  const s = span as unknown as {
    instrumentationScope?: { name?: string };
    instrumentationLibrary?: { name?: string };
  };
  return s.instrumentationScope?.name ?? s.instrumentationLibrary?.name ?? "";
}

/** True when the app opted in to exporting prompts, outputs and tool payloads. */
export function captureContentEnabled(env: Env = process.env): boolean {
  return /^(1|true|yes|on)$/i.test((env.FI_CAPTURE_CONTENT ?? "").trim());
}

/**
 * Attributes that identify the caller. eve copies the authenticated principal's
 * id here, which can be a user id or an email, so they are dropped unless the
 * app opted in to content capture. The principal *type* is kept.
 */
export const PRINCIPAL_ID_KEYS: ReadonlySet<string> = new Set([
  "agent.principal.current.id",
  "agent.principal.initiator.id",
]);

/**
 * The Future AGI destination's export policy. By default it keeps span
 * metadata (names, timing, model, tokens, tool names, status code) and redacts
 * content: prompts, system instructions, tool definitions, model output, tool
 * arguments and results, error messages, exception events, the session title,
 * and the principal ids above. FI_CAPTURE_CONTENT=1 turns all of that back on,
 * up to eve's own otel({ tracePolicy }) ceiling.
 */
export function futureAGIExportPolicy(env: Env = process.env): SpanExportPolicy[] {
  if (captureContentEnabled(env)) return [];
  return [
    {
      span: () => ({ redact: true, inputs: true, outputs: true }),
      attribute: ({ key }) => ({ emit: !PRINCIPAL_ID_KEYS.has(key) }),
    },
  ];
}

/** Forwards a span to `next` unless its instrumentation scope is dropped. */
export class ScopeFilterSpanProcessor implements SpanProcessor {
  readonly next: SpanProcessor;
  readonly dropped: ReadonlySet<string>;

  constructor(next: SpanProcessor, dropped: ReadonlySet<string> = DROPPED_SCOPES) {
    this.next = next;
    this.dropped = dropped;
  }

  onStart(span: Span, parentContext: Context): void {
    if (!this.dropped.has(scopeName(span))) this.next.onStart(span, parentContext);
  }

  onEnd(span: ReadableSpan): void {
    if (!this.dropped.has(scopeName(span))) this.next.onEnd(span);
  }

  forceFlush(): Promise<void> {
    return this.next.forceFlush();
  }

  shutdown(): Promise<void> {
    return this.next.shutdown();
  }
}

/** The OTLP/HTTP traces URL: FI_BASE_URL (default: hosted) + /tracer/v1/traces. */
export function futureAGITracesUrl(env: Env = process.env): string {
  const base = (env.FI_BASE_URL || "https://api.futureagi.com").replace(/\/+$/, "");
  return `${base}/tracer/v1/traces`;
}

/** OTLP/HTTP protobuf exporter for the Future AGI trace endpoint. */
export function futureAGIExporter(env: Env = process.env): OTLPHttpProtoTraceExporter {
  return new OTLPHttpProtoTraceExporter({
    url: futureAGITracesUrl(env),
    headers: {
      "X-Api-Key": env.FI_API_KEY ?? "",
      "X-Secret-Key": env.FI_SECRET_KEY ?? "",
    },
  });
}

/** Scope filter in front of a batching processor in front of the exporter. */
export function futureAGISpanProcessor(
  exporter: SpanExporter = futureAGIExporter(),
): ScopeFilterSpanProcessor {
  return new ScopeFilterSpanProcessor(new BatchSpanProcessor(exporter));
}

The span processor filters durable-workflow plumbing before export while retaining eve agent spans and other instrumentation you add. Using traceExporter directly would export those workflow spans too.

Both auth headers are required. The keys travel only as X-Api-Key and X-Secret-Key headers to the collector, never on a span and never to the model provider. The exporter sends uncompressed OTLP/HTTP protobuf.

4. Run and check

npx eve dev

Ask the agent something that calls a tool, then open your project under Observe → Tracing. Expand the turn to inspect its model and tool spans:

invoke_agent eve-weather-agent          turn total tokens, status
├── agent.step
│   ├── chat gpt-5.4-mini               model, provider, input and output tokens
│   └── execute_tool get_weather        tool name, call id, ERROR if the tool throws
└── agent.step
    └── chat gpt-5.4-mini

The root span is named after the agent. Each user turn starts a trace unless it inherits a local caller’s trace.

eve span	Future AGI type	What you see
invoke_agent <agent>	unknown (see Limitations)	Duration, the turn’s total input and output tokens, status
agent.step	unknown	Duration, step index
chat <model>	llm	Model, provider (the AI SDK provider id), input and output tokens, finish reason
execute_tool <tool>	tool	Tool name, tool call id, duration, error.type and ERROR status when the tool fails

A failed model call (for example a wrong model API key) shows ERROR on the chat, agent.step and invoke_agent spans.

What is captured

Data	Default	FI_CAPTURE_CONTENT=1
Span names, timing, parent links, status code	Yes	Yes
Model id, provider, token usage, finish reason	Yes	Yes
Tool name, tool call id, error.type	Yes	Yes
eve run, turn and step ids, channel name, gen_ai.conversation.id, principal type	Yes	Yes
Prompts, system instructions, tool definitions	No	Yes, up to eve’s ceiling
Model output	No	Yes, up to eve’s ceiling
Tool arguments and results	No	Yes, up to eve’s ceiling
Error messages and exception events (stack traces)	No	Yes, up to eve’s ceiling
Session title (which can contain the user’s first message)	No	Yes, up to eve’s ceiling
Principal ids (agent.principal.current.id, agent.principal.initiator.id)	No	Yes
eve’s workflow spans	Never	Never

The resource carries service.name (your package name), cloud.provider, process.runtime.name, vercel.runtime, project_name and project_type.

Privacy

Default: metadata only. The destination’s export policy uses eve’s own redaction. It removes prompts, system instructions, tool definitions, model output, tool arguments and results, error messages, exception events and the session title from what Future AGI receives, and the recipe also drops the two principal id attributes. It does not change eve’s local traces, other destinations, or Workflow run-data retention.

Opt in: FI_CAPTURE_CONTENT=1. This removes the redaction for the Future AGI destination. Content then flows up to eve’s own capture ceiling, otel({ tracePolicy }). By default eve records content only in development and for public conversations (eve docs). If eve’s capture ceiling excludes a conversation, widen it only after reviewing the privacy implications:

import { otel } from "eve/instrumentation/otel";

export default otel({
  resource: {
    project_name: process.env.FI_PROJECT_NAME ?? "eve-agent",
    project_type: "observe",
  },
  // Widens content capture for every OpenTelemetry destination, not just Future AGI.
  tracePolicy: () => ({ emit: true, recordInputs: true, recordOutputs: true }),
});

tracePolicy applies to every destination, including Vercel Agent Runs on Vercel, and eve also uses the resolved decision to pick Workflow run-data retention. Read eve’s notes on both before widening it.

With content on, inspect gen_ai.input.messages, gen_ai.output.messages, gen_ai.system_instructions, gen_ai.tool.call.arguments and gen_ai.tool.call.result in the span’s Attributes panel. Whether the Input and Output panes also display those messages depends on your Future AGI collector version.

How it behaves

• Flushing. The recipe’s processor forwards forceFlush() and shutdown() to its batch processor. Let eve manage the provider lifecycle rather than registering a second provider. A forced process termination can lose queued spans.
• Streaming. One chat span per model call, ended when the stream ends, not one span per chunk.
• Sessions. Each turn is a separate trace. Turns of the same eve session share gen_ai.conversation.id and agent.run.id. Future AGI does not group them as a session, because eve sets no session.id; filter on gen_ai.conversation.id to see a conversation.
• Retries. Do not assume exactly-once spans for retried steps.

Limitations

• The root and step spans show type “unknown”. Future AGI maps chat spans to llm and execute_tool spans to tool, but not gen_ai.operation.name=invoke_agent. agent.step reports gen_ai.operation.name=workflow, eve’s own label for a step (it is in scope eve.agent and is kept). The tree, timing and status are correct.
• Content display depends on the collector. If the Input and Output panes are empty with FI_CAPTURE_CONTENT=1, inspect the message attributes directly. Do not enable content capture just to diagnose missing metadata.
• Tokens appear twice in a trace. invoke_agent carries the turn’s totals and each chat span carries its own usage. A sum over every span of a trace counts the turn twice.
• Provider shows the AI SDK provider id, for example openai-compatible.chat.

Troubleshooting

• eve build fails with Found removed instrumentation file "agent/instrumentation.ts". The file is from a pre-0.62 guide. Move the setup into the three files above.
• No traces in the project. Check that otel.ts sets project_name on the resource (a missing project_name makes the collector reject the batch), that both FI_API_KEY and FI_SECRET_KEY are set, and that FI_BASE_URL is an origin without a path.
• Dozens of workflow.* and step.* spans. The destination uses traceExporter. Use spanProcessors: [futureAGISpanProcessor()] so the scope filter runs.
• FI_CAPTURE_CONTENT=1 but no prompts from a deployed agent. eve’s ceiling excludes the conversation. Widen otel({ tracePolicy }) as shown in Privacy.
• Cannot select model ... modelContextWindowTokens. eve needs modelContextWindowTokens on defineAgent for a model it cannot look up in the AI Gateway catalog. Unrelated to tracing.
• eve start answers 401. That is eve’s route auth, not tracing. Configure an authenticator in agent/channels/eve.ts; see eve’s authentication guide.