Explore a queue

Once a queue has items and annotators in it, this detail view is where the work actually happens. This guide walks that view using Support quality review, the queue built earlier, as the example; any queue of your own works the same way.

These guides all start from a queue you’ve already created. If you don’t have one yet, Create a queue makes the first one.

Open the queue

Go to Annotations → Queues and click the Support quality review row to open it.

Read the header

The header carries the queue’s name, a status badge next to it, progress underneath, and a row of actions on the right, covered below.

The badge reads Draft, Active, Paused, or Completed. It’s the one place to check whether the queue is unlocked for annotating before you try to open an item.

If you have items assigned to you, you get two progress bars: your own progress first, then the queue’s overall progress with a breakdown of how many items are pending, in progress, in review, or skipped.

The five tabs

Below the header, the view splits into five tabs:

• Items, the list of items in the queue and where you open one to annotate it
• Settings, the queue’s configuration, managers only
• Analytics, performance across the queue
• Agreement, how consistently annotators score the same items
• Rules, automation rules that feed items into the queue, managers only

Settings and Rules only show up if you’re a manager on the queue. Everyone else sees Items, Analytics, and Agreement.

The toolbar

Four actions live in the header toolbar. Two of them switch labels depending on the queue’s state, and which ones you see at all depends on your role:

• Activate shows up for managers only, and only while the queue isn’t already active
• Export opens a menu with Download and Export to Dataset. It’s grayed out until the queue has items in it
• Review Items is for reviewers and managers, and shows up once the queue has items and is active or completed. If the queue doesn’t require review, this button reads View Submissions instead
• Start Annotating opens the annotation workspace for anyone who can annotate, picking the next available item for you. Once the queue is complete and has skipped items left, this button reads Resume Skipped instead. Opening an item straight from the Items tab lands you in that same workspace, just on the item you clicked instead of the next one in line

The two rules for annotating an item

• You can only annotate while the queue is active, except to resume skipped items once it’s completed. Try to open an item outside that state and you’re told to manage the status from the Settings tab first. If you’re a manager, the toolbar’s Activate button does the same thing in one click whenever the queue isn’t already active; use Settings for any other status change, or if you don’t see the button
• You can only open an item assigned to you, unless auto-assign is on, or you’re a manager or reviewer on the queue. Auto-assign is a checkbox in the queue’s Settings tab, the same one you set when you built the queue. For plain annotators, an item assigned to someone else stays closed even if you can see it in the list

Dive deeper