Output types & scoring

What a result carries

Every evaluation run produces a result for each row, span, or test input it scores.

Part	What it is	When present
Value	The verdict, score, or label, shaped by the template’s output type (the three types below)	Always
Reason	A plain-language explanation of the verdict from the evaluator model	LLM-as-Judge and Agent Evaluator evals; Code Evals have no evaluator model, so no reason

The reason is worth reading before anything else when a result surprises you: instead of re-reviewing the response manually, you read why the evaluator model called it the way it did.

The three output types

Whatever shape the value takes, every result resolves to an underlying score between 0 and 1. The two scoring settings below act on that number.

Output type	The value	How it becomes a verdict	Typical templates
Pass/Fail	Pass or fail	The score checked against the pass threshold	Toxicity, PII detection, format checks
Percentage	A graded score, shown as a percentage	Passes when the score clears the threshold	Groundedness, relevance, completeness
Deterministic choices	One label from the set the template defines	Each label carries a score via choice scores	Tone, language, intent

Built-in templates ship with their output type fixed; a custom eval picks one when you create it.

Scoring settings

Two template settings turn a raw output into a verdict you can gate on:

Setting	What it does	Default
Pass threshold	A score at or above the threshold counts as a pass	The midpoint, 0.5
Choice scores	Maps each choice label to a score, for example {"Yes": 1.0, "No": 0.0}, so categorical results still roll up into numbers	Set per template

Both live in the eval’s configuration: set them when you create a custom eval or attach one to a run.

Aggregates

A single result judges one response; the platform also rolls results up per eval:

Aggregate	What it tells you	Applies to
Pass rate	The share of scored rows that passed	Pass/Fail outputs
Average score	The mean score across rows	Percentage outputs

Aggregates are tracked per template per run, so the same eval gives you comparable quality numbers across datasets, runs, and template versions.

Where results land

Surface	How results show
Datasets and experiments	A result column per eval, one value per row, with an optional reason column alongside
Traces	A span’s eval results in the trace detail view in Observe, with the flagged input when error localization ran
Playground	Value and reason per test, as soon as the run finishes
Code	The SDK returns the same result as an object; see SDK & API

Keep exploring