SDK & API

Three ways to work with annotations

The dashboard is where you set up and run a campaign: build a queue, attach labels, add annotators, and watch it through to completion.

This page covers the Python SDK and the REST API. The Python SDK’s fi.queues.AnnotationQueue client covers the queue lifecycle end to end, from a script. It:

• creates queues
• creates labels
• adds and assigns items
• submits annotations
• reads progress and analytics
• exports

The REST API covers the same ground, plus every other endpoint the platform exposes. Both surfaces can also score a source directly, without a queue involved at all: a trace, span, session, dataset row, call execution, or prototype run you want to annotate without the queue workflow around it, via create_score() in Python or Create Score over REST.

Install and authenticate

pip install futureagi

from fi.queues import AnnotationQueue

client = AnnotationQueue(
    fi_api_key="YOUR_API_KEY",
    fi_secret_key="YOUR_SECRET_KEY",
)

You can also set FI_API_KEY and FI_SECRET_KEY as environment variables and drop both arguments; the client picks them up automatically. Find both under Settings → API Keys in the platform.

An end-to-end example

Creating Support quality review, pushing two traces into it, checking progress, then pulling the completed results back out:

queue = client.create(name="Support quality review", instructions="Rate response quality 1-5")

client.add_items(queue.id, items=[
    {"source_type": "trace", "source_id": "trace_abc123"},
    {"source_type": "trace", "source_id": "trace_def456"},
])

progress = client.get_progress(queue.id)
print(f"{progress.completed} of {progress.total} done")

results = client.export(queue.id, export_format="json", status="completed")

Job to method to endpoint

Each job below has a Python method and a REST endpoint that do the same thing. Full parameter tables live on the linked SDK pages, not here.

Job	Python SDK	REST API
Create a label	create_label()	Create Label
Create a queue	create()	Create Queue
Add items	add_items()	Add Items
Submit annotations for a queue item	submit_annotations()	Submit Annotations
Score a source directly	create_score()	Create Score
Read progress	get_progress()	Get Progress
Export	export()	Export

Keep exploring