Labels

A reusable question with a fixed answer type

A label is a reusable question plus the answer type it accepts. Attach it to a queue and every annotator working that queue answers the same question, the same way, and each answer lands as a score. The full object model connecting labels, queues, and scores lives in Understanding Annotation; this page is about the label on its own.

Three rules that follow

A label’s type is locked the moment you create the label and can’t be changed afterward, though you can still edit its name, description, or settings. You can’t turn a numeric label into a categorical one, for example: the type fixes both the control the annotator sees and the shape of the value written into every resulting score. Picked the wrong type? Create a new label with the right type instead of trying to change this one.

A label belongs to the org you created it in, not to any single queue, so editing it changes every queue it’s attached to. A queue attaches an existing label rather than copying it, so editing a label’s name, description, or settings changes what every attached queue shows and validates against from that point on. Scores already submitted aren’t touched; the edit applies to answers submitted after it.

A queue needs at least one label to exist. You can’t create or save a queue with zero labels attached. The label is what turns a pile of items into something answerable.

flowchart TD
accTitle: What a label's type fixes, and the queues it's attached to
accDescr: A label carries a fixed type. The type sets the control an annotator sees and the shape of the value stored in every resulting score. The same label attaches to multiple queues, each of which needs at least one label to exist.
L["Label · Response quality · type categorical, locked"] --> C["Annotator's control · the label's option list"]
L --> V["Value shape in every score · the selected option(s)"]
L -->|attached to| Q1["Queue · Support quality review · needs 1+ label"]
L -->|attached to| Q2["Queue · Onboarding review · needs 1+ label"]

Five types, five kinds of judgement

• Categorical collects one option from a list you define, or several if you allow multiple selection, best for a defect category, a sentiment, or an escalation reason
• Numeric collects a number within the range you set, best for relevance on a 1-to-10 scale or a quality score out of 100
• Text collects free-text feedback in the annotator’s own words, best for detail that doesn’t reduce to an option or a number
• Star Rating collects a star count on a scale you choose when creating the label, from 1 up to 10 stars, best for a fast overall impression
• Thumbs Up/Down collects a single up or down call, best for a binary pass or fail judgement

The settings each type requires and the exact validation applied to a submitted value live on Label types & values.

Picking a type

Match the type to the shape of the judgement, not the topic:

If you need	Pick
The answer to be one or more of a few known outcomes	Categorical
The answer to be a quantity	Numeric
The answer explained, not selected	Text
A quick, coarse gut check	Star Rating
The answer to be strictly one of two	Thumbs Up/Down

If an item needs more than one kind of judgement, attach more than one label to the queue rather than stretching a single label to cover two jobs.

Why it matters

Standardizing on a small set of labels (one for quality, one for tone) keeps scores comparable across teams. Response quality, for instance, is a categorical label with three options, Good, Needs work, Wrong, so everyone answering it is choosing from that exact same set, not inventing their own scale.

Keep exploring