Track prompt performance
See how each version of a prompt behaves on real traffic
The Metrics tab in the editor rolls latency, tokens, and cost up per version for support-agent, so you can compare two versions head to head instead of reading traces one at a time.
Before you start: Metrics only pick up generations that carry a reference to the prompt template they came from. Wire that up first, as covered in Log prompt templates; this guide picks up once that’s in place.
Run the prompt first
The Metrics tab stays disabled until the prompt has produced at least one output. Open it too early and the tab shows why, in its own tooltip: “You need to submit at least one prompt and get an output before accessing the metrics.” Run support-agent once and the tab unlocks.
View per-version metrics
Open the prompt and switch to Metrics
In the editor, open support-agent and click the Metrics tab. It splits into two sub-tabs: Metrics, the per-version table below, and Linked Traces, the individual traces behind those numbers.
Read the per-version medians
The table lists one row per version, aggregated across every trace recorded for it:
| Metric | What it tells you |
|---|---|
| Median Latency | Typical time for the model to produce a response |
| Median Input Tokens | Typical size of the prompt sent to the model |
| Median Output Tokens | Typical length of the model’s reply |
| Median Cost | Typical cost per generation for this version |
| No. of traces | How many times this version was called |
| First Used | When this version was first called |
| Last Used | When this version was most recently called |
| Label Name | Which label, if any, points at this version, useful for telling which one Production is live on |
If a version you expect doesn’t show up, or its trace count looks lower than it should, its generations most likely aren’t carrying the template reference. Go back to Log prompt templates and check the instrumentation.
Drill into the traces behind a number
Switch to the Linked Traces sub-tab to see the individual traces that rolled up into those numbers, useful when a median looks off and you want to check what actually produced it.
Decide if a change helped
Pick a metric, then compare it across two versions. If support-agent v3’s median latency comes in lower than v2’s, the change helped. If median cost jumps right after you lengthen the system message, that’s the number that tells you why.
Dive deeper
Questions & Discussion