Traces Are Noisy or Incomplete

Traces you don't recognize show up in this project, or a trace's Type/Cost/Session/User is blank even though the LLM calls inside it ran. Usual causes: a process-wide instrumentor, a hand-built root span, or an oversized payload.

Symptom

A trace shows up, but something about it is wrong in one of two directions: either it’s not yours, or it’s missing the fields that make it useful. Both usually trace back to how the root span (the top row of the trace) was created, not to a bug in your data.

  • Rows appear in this project that don’t match your own code’s input/output shape: a different call signature, a different kind of prompt.
  • A trace’s Type shows “unknown”, Cost shows $0.00, and it’s missing from both the Sessions and Users tabs, even though you can see LLM calls with a real model and cost when you open the span tree underneath it.
  • A trace is missing entirely, or your exporter logs a 413 Request Entity Too Large or RESOURCE_EXHAUSTED.

Quick checks

  • Does more than one workflow using the same framework (e.g. LangChain) run inside the same process? Instrumenting one wires up all of them.
  • Did you create the top-level span yourself (tracer.start_as_current_span(...)) instead of letting the framework’s auto-instrumentor create the root?
  • Does any single call send an unusually large prompt, tool result, or document as an attribute?

Causes and fixes

CauseWhat you seeFix
One instrumentor patches the whole processTraces from code you don’t own show up in this projectSomeFrameworkInstrumentor().instrument(tracer_provider=...) patches that framework’s callback/client machinery for the entire process, not just the module that called it. A second instrument() call elsewhere in the same process is a no-op and reuses the first tracer_provider. Give unrelated workflows their own process, or their own tracer_provider registered before either one runs.
Suppressing drops spans, it doesn’t redirect themYou suppressed the unrelated calls and now they don’t show up anywhere, not just in this projectsuppress_tracing() drops spans created in its block. It never moves them to another project. If that code still needs to be observable, give it its own tracer_provider/project instead of suppressing it.
Manual root span never gets the standard fieldsTrace Type is “unknown”, no Session/UserAn auto-instrumented span picks up session.id/user.id from using_session/using_user automatically. A root span you create yourself doesn’t get gen_ai.span.kind, session.id, or user.id for free: set them explicitly on it (see below).
The row’s Cost/Model always reflect the root spanCost and Model stay $0.00 / blank on the row even after Session and User are fixedThe trace table’s Cost, Model, and Tokens columns read the root span only. A hand-built root that never sets those attributes shows blank on the row even though the LLM call underneath it has both; open the span tree to see them there.
Oversized payloadExporter logs 413 Request Entity Too Large / RESOURCE_EXHAUSTED, or a span silently never landsA single span over 16 MiB is rejected outright. Cap attribute size with OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT or span_limits so no one attribute can grow that large.

Fix it: set the standard fields on a manual root span

from fi_instrumentation import using_session, using_user
from fi_instrumentation.fi_types import FiSpanKindValues, SpanAttributes

with using_session(session_id), using_user(user_id):
    with tracer.start_as_current_span("agent_turn") as span:
        result = agent.invoke({"messages": [{"role": "user", "content": question}]})

        span.set_attribute(SpanAttributes.GEN_AI_SPAN_KIND, FiSpanKindValues.AGENT.value)
        span.set_attribute(SpanAttributes.SESSION_ID, session_id)
        span.set_attribute(SpanAttributes.USER_ID, user_id)

Set all three explicitly, even though using_session/using_user also tag a FITracer span automatically; a plain OpenTelemetry tracer does not get them, so setting them here works either way.

Fix it: keep an unrelated workflow out of this project

from fi_instrumentation import suppress_tracing

with suppress_tracing():
    result = other_workflow.invoke(...)   # never traced, anywhere, in any project

Only reach for this if that workflow doesn’t need to be observable at all. If it does, register a second tracer_provider for it instead: see Instrument your project.

async with suppress_tracing(): isn’t supported: it raises a TypeError and can leave tracing suppressed past the block. Use the synchronous with, even inside an async def.

Fix it: cap oversized attributes

export OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT=100000

or in code, once at registration:

from fi_instrumentation import register, SpanLimits

trace_provider = register(
    project_name="my_project",
    span_limits=SpanLimits(max_attribute_length=100000),
)

Diagnostic commands

Print each span’s name and attributes as they’re created, so you can see exactly which span has what before anything is exported:

from opentelemetry.sdk.trace.export import SimpleSpanProcessor, SpanExporter

class PrintAttributes(SpanExporter):
    def export(self, spans):
        for span in spans:
            print(span.name, dict(span.attributes))
        return 0

trace_provider.add_span_processor(SimpleSpanProcessor(PrintAttributes()))

If the root span’s print shows no gen_ai.span.kind/session.id/user.id, it never had them; that’s the manual-root case, not a UI bug. If you see spans printed for code you don’t recognize, that’s the shared-instrumentor case.

Minimal smoke test

Run one request through end to end, then check three things on that trace: it’s the only new row in the project (no unrelated code’s calls alongside it), its Type/Session/User are populated on the row, and it landed at all if the request included an unusually large input or output.

Escalate

If a trace still looks wrong after the fixes above, contact support@futureagi.com with the project_name, the trace ID, and which of the three symptoms above you’re still seeing.

Prevent recurrence

  • Give each distinct framework-based workflow its own process or its own tracer_provider before you run more than one in the same service.
  • If you build a root span by hand, treat gen_ai.span.kind, session.id, and user.id as a fixed checklist to set on it every time, not a maybe.
  • Set a size cap once at registration, rather than after you hit the 16 MiB limit in production.

Next steps

Was this page helpful?

Questions & Discussion