Adding Annotations to your Spans

Label spans with custom tags, human feedback, and notes using the bulk-annotation API.

Tip

Looking for the new unified Annotations system? Check out the Annotations documentation for annotation queues, managed workflows, and the Scores API.

About

Traces show what happened but not whether the result was correct, helpful, or safe. Annotations close that gap by attaching labels, scores, notes, and human feedback directly to spans. The /tracer/bulk-annotation/ API lets this be done programmatically, at scale, across hundreds of spans in a single request. Annotated spans can then be filtered by quality, exported as golden datasets, or used in RLHF workflows.

When to use

Label data for filtering and analysis: Tag spans with custom criteria so they can be searched and grouped in the dashboard.
Build golden datasets: Annotate high-quality examples for AI training and fine-tuning.
Add human feedback: Attach scores, thumbs up/down, or notes to spans for RLHF and evaluation workflows.
Enrich trace context: Add custom events and notes to spans for richer debugging.

How to

Create an annotation label

Annotation labels must be created before using the API. See the Labels guide for how to create and configure labels (text, numeric, categorical, star, thumbs up/down).

Fetch your annotation label ID

Before attaching annotations via the API, retrieve the annotation_label_id for the label you created. Use the /tracer/get-annotation-labels/ endpoint.

import requests

BASE_URL = "https://api.futureagi.com"
headers = {                       # API-key or JWT, as described above
    "X-Api-Key":     "<API_KEY>",
    "X-Secret-Key":  "<SECRET_KEY>",
    "Content-Type":  "application/json",
}

resp = requests.get(f"{BASE_URL}/tracer/get-annotation-labels/?project_id=<PROJECT_ID>", headers=headers, timeout=20) # replace <PROJECT_ID> with your project id if you want to get the label for a specific project
resp.raise_for_status()

label_id = resp.json()["result"][0]["id"]   # first label in your project, remove the index if you have more than one label
print("Annotation-label ID:", label_id)

The response contains a list of all labels in your project; each item includes id, name, type, and other metadata.

Send annotations via the API

Use the /tracer/bulk-annotation/ endpoint to add annotations to one or more spans. Authenticate with your API key and Secret key.

POST https://api.futureagi.com/tracer/bulk-annotation/

   X-Api-Key: <YOUR_API_KEY>
   X-Secret-Key: <YOUR_SECRET_KEY>

All requests must also include Content-Type: application/json.

The records array targets one or more spans. Inside each record you can add new annotations and notes, update existing annotations (matched by annotation_label_id + annotator_id), and add notes (duplicates are silently ignored).

{
  "records": [
    {
      "observation_span_id": "<SPAN_ID>",     // span to annotate
      "annotations": [
        {
          "annotation_label_id": "lbl_123",          // your label id
          "annotator_id": "human_annotator_2",       // who is annotating
          "value": "good"                            // TEXT label
        },
        {
          "annotation_label_id": "lbl_123",
          "annotator_id": "human_annotator_2",
          "value_float": 4.2                         // NUMERIC label
        },
        {
          "annotation_label_id": "lbl_123",
          "annotator_id": "human_annotator_3",
          "value_bool": true                         // THUMBS label
        },
        {
          "annotation_label_id": "lbl_123",
          "annotator_id": "human_annotator_4",
          "value_str_list": ["option1", "option2"]   // CATEGORICAL label
        }
      ],
      "notes": [
        {
          "text": "First note",
          "annotator_id": "human_annotator_1"
        }
      ]
    },
  ]
}

Supported value keys per label type:

Label Type	Field to Use	Example Value
Text	`value`	`"Loved the answer"`
Numeric	`value_float`	`4.2`
Categorical	`value_str_list`	`["option1", "option2"]`
Star rating	`value_float`	`4.0` (1–5)
Thumbs up/down	`value_bool`	`true` or `false`

End-to-end example

A complete example showing label lookup, payload construction, and the annotation request.

#!/usr/bin/env python3
import json, requests

from datetime import datetime
from rich import print as rprint
from rich.console import Console
from rich.table import Table

BASE_URL      = "https://api.futureagi.com"
FI_API_KEY    = "<YOUR_API_KEY>"
FI_SECRET_KEY = "<YOUR_SECRET_KEY>"

console = Console()

def headers():
    return (
        {
            "X-Api-Key": FI_API_KEY,
            "X-Secret-Key": FI_SECRET_KEY,
            "Content-Type": "application/json",
        }
    )

def get_first_label_id():
    resp = requests.get(f"{BASE_URL}/tracer/get-annotation-labels/", headers=headers(), timeout=20)
    resp.raise_for_status()
    label = resp.json()["result"][0]
    console.log(f"Using label: {label['name']} ({label['type']})")
    return label["id"]

def build_payload(span_id, label_id):
    ts = datetime.utcnow().isoformat(timespec="seconds")
    return {
        "records": [
            {
                "observation_span_id": span_id,
                "annotations": [
                    {"annotation_label_id": label_id, "annotator_id": "human_a", "value": "good"},
                    {"annotation_label_id": label_id, "annotator_id": "human_a", "value_float": 4.2},
                ],
                "notes": [{"text": "First note " + ts, "annotator_id": "human_a"}],
            }
        ]
    }

def pretty(resp_json):
    table = Table(title="Bulk-Annotation Result", show_header=True, header_style="bold cyan")
    table.add_column("Key"); table.add_column("Value", overflow="fold")
    for k, v in resp_json.items():
        table.add_row(k, json.dumps(v, indent=2) if isinstance(v, (dict, list)) else str(v))
    console.print(table)

if __name__ == "__main__":
    SPAN_ID  = "<SPAN_ID>"
    payload  = build_payload(SPAN_ID, get_first_label_id())
    rprint({"payload": payload})

    resp = requests.post(f"{BASE_URL}/tracer/bulk-annotation/", headers=headers(), json=payload, timeout=60)
    resp.raise_for_status()
    pretty(resp.json())

#!/usr/bin/env ts-node
import axios from "axios";

const BASE_URL = "https://api.futureagi.com";
const SPAN_ID  = "<SPAN_ID>";

// Choose ONE auth method
const FI_API_KEY  = "<YOUR_API_KEY>";
const FI_SECRET_KEY = "<YOUR_SECRET_KEY>";

// ────────────────────────────

function headers(): Record<string, string> {
    return {
      "X-Api-Key": FI_API_KEY,
      "X-Secret-Key": FI_SECRET_KEY,
      "Content-Type": "application/json",
    };
}

async function getFirstLabelId(): Promise<string> {
  const resp = await axios.get(`${BASE_URL}/tracer/get-annotation-labels/`, {
    headers: headers(),
    timeout: 20000,
  });
  const label = resp.data.result[0];
  console.log(`Using label: ${label.name} (${label.type})`);
  return label.id;
}

function buildPayload(spanId: string, labelId: string) {
  const ts = new Date().toISOString().slice(0, 19);

  const recordNew = {
    observation_span_id: spanId,
    annotations: [
      { annotation_label_id: labelId, annotator_id: "human_annotator_1", value: "good" },
    ],
    notes: [
      { text: "First note " + ts, annotator_id: "human_annotator_1" },
    ],
  };

  return { records: [recordNew] };
}

async function main() {
  try {
    const labelId = await getFirstLabelId();
    const payload  = buildPayload(SPAN_ID, labelId);

    console.log("\n──── REQUEST PAYLOAD ────");
    console.dir(payload, { depth: null });

    const resp = await axios.post(`${BASE_URL}/tracer/bulk-annotation/`, payload, {
      headers: headers(),
      timeout: 60000,
    });

    console.log("\n──── RESPONSE ────");
    console.dir(resp.data, { depth: null });
  } catch (err: any) {
    if (err.response) {
      console.error(`HTTP ${err.response.status}`);
      console.error(err.response.data);
    } else {
      console.error("Error:", err.message);
    }
    process.exit(1);
  }
}

main();

curl -X POST https://api.futureagi.com/tracer/bulk-annotation/ \
-H "X-Api-Key: <YOUR_API_KEY>" \
-H "X-Secret-Key: <YOUR_SECRET_KEY>" \
-H "Content-Type: application/json" \
-d '{"records": [{"observation_span_id": "<SPAN_ID>", "annotations": [{"annotation_label_id": "<LABEL_ID>", "annotator_id": "human_annotator_1", "value": "good"}]}]}'

Key concepts

Response object

Every call returns a top-level boolean status and a nested result object:

Field	Type	Meaning
status	boolean	true if the request itself was processed (even if some records failed).
result.message	string	Human-readable summary.
result.annotationsCreated	number	How many annotations were created across all records.
result.notesCreated	number	How many notes were created across all records.
result.succeededCount	number	Number of records that were applied without errors.
result.errorsCount	number	Number of records that had at least one error.
result.errors	array	Per-error details (see below).

Error objects

Each element in result.errors contains:

Field	Type	Example	Description
recordIndex	number	1	Position of the offending record in the records array (0-based).
spanId	string	”45635513961540ab”	The span that failed.
annotationError	string	”Annotation label “axdf” does not belong to span’s project”	Error message for the annotation operation (optional).
noteError	string	”Duplicate note”	Error message for the note operation (optional).

Adding Annotations to your Spans

About

When to use

How to

Create an annotation label

Fetch your annotation label ID

Send annotations via the API

End-to-end example

Key concepts

Next Steps

Set Up Tracing

Add Attributes & Metadata

In-line Evaluations

Auto Instrumentation

Questions & Discussion

FutureAGI AI Assistant

About

When to use

How to

Create an annotation label

Fetch your annotation label ID

Send annotations via the API

End-to-end example

Key concepts

Next Steps

Set Up Tracing

Add Attributes & Metadata

In-line Evaluations

Auto Instrumentation

Questions & Discussion