Automate item intake

A rule checks new candidates against conditions you set and adds the matches to a queue on its own, so nobody has to go looking for fresh items to hand out. This walks through building one for Support quality review, the running example from Create a queue, running it once to see what it catches, and what actually happens once you trigger it. Creating, editing, deleting, and running a rule are all manager work: if you’re not a manager on the queue, following these steps just gets you an error. If you’d rather add items yourself, see Add items.

Create a rule

Only queue managers can create or run rules. If you’re not one, step 1 below returns “Only queue managers can manage automation rules.” instead of opening the form.

From Support quality review’s Rules tab, click Add Rule in the top right to start a new rule.

1. Give it a name

2. Choose a Source type to say what kind of candidate the rule looks at, one of:

   • Dataset Row
   • Trace
   • Span
   • Session
   • Simulation

3. Set the Trigger, one of:

   • Manually: the rule never fires on its own, someone has to run it every time
   • Every hour, Daily, Weekly, or Monthly: puts it on a recurring schedule instead

4. Pick the specific target the rule reads from, one of:

   • Dataset
   • Project
   • Agent Definition

   If the queue is already scoped to a dataset or project, that field is locked and reads “Locked by this queue”

5. Add the conditions that decide which candidates match under Conditions. The available fields and operators depend on the Source type you picked

6. Click Create Rule. It stays disabled until the rule has a name and a source

Once a rule exists, click its row to open Edit Automation Rule and change its name, source type, conditions, or trigger. Delete is the x at the end of the row; it asks for confirmation first.

Run it once before you trust it

Whatever trigger you picked, click Run Now on the rule’s row in the Rules tab to see what it catches from its source right now, before you let it run unattended. The first run always scans the whole backlog against the conditions, whether it fires by schedule or because you clicked Run Now. After that first run, a scheduled rule only rescans what’s new since it last ran, so Run Now on a rule that’s already fired once is a smaller check, not a full rescan; a Manually-triggered rule has no schedule to fall back on, so every run stays a full rescan. Run Now stays disabled until the rule is enabled with the Enabled switch in the rule’s row; its tooltip reads “Enable this rule before running it”. Click Run Now again while a run is still going and it’s refused with “A run is already in progress for this rule”.

When a rule runs

This is the part that trips people up:

• A scheduled rule (Every hour, Daily, Weekly, Monthly) doesn’t fire at the exact minute its trigger implies, so treat the trigger as “within about an hour of,” not “on the dot”
• Running a rule yourself either reports what it added in the toast right away, or shows “We’re preparing your data” and finishes in the background
• When a run finishes in the background, the person who ran it, the rule’s creator, and every manager on the queue get an email. Scheduled runs don’t send it

New items don’t arrive silently

Even without that email, annotators find out. Everyone gets a new-item email at most once an hour, plus a daily summary at their own local digest hour, unless they’ve snoozed notifications. Whatever a rule adds to Support quality review reaches annotators on both cadences, so a rule firing while you’re not watching still gets to the people who need to work the items.

Dive deeper