Scores

Scores provide a unified annotation model that can be used independently of queues to annotate any source entity.

create_score

Creates a single score with upsert semantics.

def create_score(
    self,
    source_type: str,
    source_id: str,
    label_id: Optional[str] = None,
    value: Any = None,
    *,
    label_name: Optional[str] = None,
    score_source: str = "api",
    notes: Optional[str] = None,
    timeout: Optional[int] = None,
) -> Score

• Arguments:
  • source_type (str): Source entity type — "trace", "observation_span", "trace_session", "call_execution", "prototype_run", or "dataset_row".
  • source_id (str): UUID of the source entity.
  • label_id (Optional[str]): UUID of the annotation label.
  • value (Any): Annotation value (str, float, bool, or list depending on label type).
  • label_name (Optional[str]): Name of the annotation label (alternative to label_id).
  • score_source (str): Origin of the score — "human", "api", or "auto". Defaults to "api".
  • notes (Optional[str]): Free-text notes.
  • timeout (Optional[int]): Request timeout in seconds.
• Returns:
  • Score instance

create_scores

Creates multiple scores on a single source entity in one request.

def create_scores(
    self,
    source_type: str,
    source_id: str,
    scores: List[Dict[str, Any]],
    *,
    notes: Optional[str] = None,
    timeout: Optional[int] = None,
) -> Dict[str, Any]

• Arguments:
  • source_type (str): Source entity type.
  • source_id (str): UUID of the source entity.
  • scores (List[Dict[str, Any]]): List of dicts, each with label_id, value, and optionally score_source.
  • notes (Optional[str]): Shared free-text notes.
  • timeout (Optional[int]): Request timeout in seconds.
• Returns:
  • Dict[str, Any]

get_scores

Gets all scores for a given source entity.

def get_scores(
    self,
    source_type: str,
    source_id: str,
    *,
    timeout: Optional[int] = None,
) -> List[Score]

• Arguments:
  • source_type (str): Source entity type.
  • source_id (str): UUID of the source entity.
  • timeout (Optional[int]): Request timeout in seconds.
• Returns:
  • List[Score]