Observability in LLM-based applications relies on a structured framework that captures execution details at different levels of granularity. Each request follows a well-defined path, where individual operations are recorded, grouped into execution flows, and organized for broader analysis. This structured approach enables teams to track model performance, debug failures, and optimize system efficiency.

​
Spans

A Span represents a single operation within an execution flow, recording input-output data, execution time, and errors. Each span provides insight into specific steps such as:

  • LLM Calls – Capturing model invocation, prompt processing, and response generation.
  • Retrieval Operations – Logging queries made to external databases or indexes.
  • Tool Executions – Tracking API calls and function invocations.
  • Error Handling – Recording failures, timeouts, and system issues.

Spans provide fine-grained visibility into each operation, allowing teams to identify where delays, errors, or inefficiencies originate.

​
Traces

A Trace connects multiple spans to represent the full execution flow of a request. It provides a structured view of how different operations interact within an LLM-powered system. Traces help teams:

  • Analyze dependencies between retrieval, inference, and tool execution.
  • Identify performance bottlenecks by measuring latency across spans.
  • Debug unexpected behaviors by tracing execution paths from input to output.

For instance, a trace for an AI-driven search system may include:

  1. A retrieval span fetching relevant documents.
  2. An LLM span generating a response.
  3. A tool execution span calling an external API.

By correlating these spans within a trace, teams can reconstruct the entire request flow, making it easier to analyze system behavior and optimize workflows.

​
Projects

A Project provides a structured way to manage multiple traces, ensuring observability is organized across different applications, use cases, or deployments. Projects allow teams to:

  • Segment and categorize observability data for different LLM-powered applications.
  • Compare model versions to track improvements in accuracy and performance.
  • Filter and analyze execution trends across multiple traces.

For example, an organization might maintain separate projects for:

  • Customer Support AI – Handling traces related to automated support queries.
  • Content Generation AI – Managing traces for LLM-powered writing assistants.
  • Legal AI Assistant – Tracking execution flows for contract analysis tasks.

By structuring observability in this way, teams can effectively monitor, compare, and optimize LLM-powered applications at scale.

Understanding ObservabilitySpans