Understanding Error Feed

What an issue is

Error Feed reads your traces and turns the problems it finds in them into issues you triage.

Nothing marks a trace as failing beforehand. Error Feed samples traces at whatever rate the project is set to, reads each sampled trace in full, and decides for itself whether something went wrong in it. A trace is “failing” only in the sense that a scan found at least one problem in it, which is why raising the sampling rate surfaces more issues: it isn’t finding more failures, it’s reading more traces.

An issue isn’t one such trace. It’s the one problem behind many of them. When ten traces go wrong the same way, the feed doesn’t hand you ten rows to read one by one, it hands you the single issue they all point at, and that issue is what you work.

What an issue carries

Every issue in the feed carries the same things, and each one is there to help you decide what to do about it:

• A title, naming the problem in a line
• A category and a group, the two labels that place the problem in the error taxonomy
• A fix layer, the part of your system the fix belongs in
• A severity and a status, the two independent axes covered in Severity & Status
• An assignee, once someone picks it up
• How often and how widely it happened: the number of times it fired, the number of traces affected, and the number of users behind those traces
• When it started and when it last happened
• The evidence behind it: the traces, spans, and sessions that contributed, so you can open the exact span rather than hunting through a trace

Issues also come from a failing eval, not only from a scan. Those group by the eval that failed, and the feed shows the eval’s name where a scan issue shows its group.

For example, ten traces that all call the wrong tool for a refund lookup surface as one cluster: title “Wrong tool selected for refund lookup”, group Tool Failures, fix layer Tools, 34 total events across 12 unique traces and 9 users affected, first seen 09:14 and last seen 14:02.

Fix layers: where the fix belongs

A fix layer is one of Prompt, Tools, Orchestration, or Guardrails. It’s the product’s actual answer to “so what do I do about this”: it names where in your system the fix belongs, not just what went wrong. Scanner clusters always carry one, taken straight from the finding. Eval clusters carry one where it can be determined: a best-effort step tries to infer it, and it’s left unset when that step can’t. The error taxonomy is the reference for which specific error types map to which fix layer.

Fix layer rides on the cluster itself, so it’s also a live filter in the feed, letting you work through everything that needs a prompt change before you touch anything that needs an orchestration change.

Why it matters

Working one issue instead of a thousand traces is what makes the feed usable at scale. A single misbehaving tool can fail on every call for an hour and put a problem in thousands of traces; read them one at a time and you’re reading the same failure a thousand times, work the issue and you fix it once. And because the fix layer sits on the issue itself, the feed tells you what to change before you’ve opened a single trace inside it.

Keep exploring