Experiments

Reference for client.experiments in the Future AGI Python and TypeScript SDKs: experiments.create, experiments.get, experiments.list.

client.experiments on fi.Client (Python) and client.experiments on Client from @future-agi/sdk (TypeScript). The models these methods return are on Datasets & evals response models; how arguments, pagination, errors and strict mode work is in the overview.

Methods

MethodDescription
experiments.createCreate and start an experiment.
experiments.getReturn an experiment by its identifier.
experiments.listList experiments in the current workspace.

client.experiments

experiments.create

Create and start an experiment. The new experiment’s id is returned inside the result message as experiment_id=\<uuid>; pass it to get_experiment to follow the run.

POST /model-hub/experiments/v2/

MCP tool: create_experiment

result = client.experiments.create(
    name="your-name",
    dataset_id="your-dataset-id",
    prompt_config=[],
    user_eval_metrics=[],
)
const result = await client.experiments.create({
  name: "your-name",
  dataset_id: "your-dataset-id",
  prompt_config: [],
  user_eval_metrics: [],
});
Signature
client.experiments.create(
    *,
    name: str,
    dataset_id: str,
    prompt_config: list[dict[str, Any]],
    user_eval_metrics: list[dict[str, Any]],
    column_id: str | None | NotGiven = NOT_GIVEN,
    experiment_type: Literal['llm', 'tts', 'stt', 'image'] | None | NotGiven = NOT_GIVEN,
    extra_query: dict[str, Any] | None = None,
    extra_body: dict[str, Any] | None = None,
    options: RequestOptions | None = None,
) -> str
client.experiments.create(params: ExperimentsCreateParams, options?: RequestOptions): Promise<string>

export interface ExperimentsCreateParams {
  name: string;
  dataset_id: string;
  prompt_config: Array<Record<string, unknown>>;
  user_eval_metrics: Array<Record<string, unknown>>;
  column_id?: string | null;
  experiment_type?: "llm" | "tts" | "stt" | "image" | null;
}

Parameters

name string Required
dataset_id string Required
prompt_config object[] Required
user_eval_metrics object[] Required
column_id string Optional
experiment_type string Optional
Allowed values:
llmttssttimage

Returns

string.

experiments.get

Return an experiment by its identifier.

GET /model-hub/experiments/v2/{experiment_id}/

MCP tool: get_experiment

result = client.experiments.get("your-experiment-id")
const result = await client.experiments.get("your-experiment-id");
Signature
client.experiments.get(
    experiment_id: str,
    *,
    extra_query: dict[str, Any] | None = None,
    options: RequestOptions | None = None,
) -> models.ExperimentDetailV2
client.experiments.get(experimentId: string, options?: RequestOptions): Promise<models.ExperimentDetailV2>

Parameters

experiment_id string Required

Positional argument experiment_id.

Returns

ExperimentDetailV2.

experiments.list

List experiments in the current workspace.

GET /model-hub/experiments/v2/list/

Paginated: returns a pager (how pagination works) · MCP tool: list_experiments

for item in client.experiments.list():
    print(item)
for await (const item of client.experiments.list()) {
  console.log(item);
}
Signature
client.experiments.list(
    *,
    created_at: str | None | NotGiven = NOT_GIVEN,
    status: str | None | NotGiven = NOT_GIVEN,
    dataset_id: str | None | NotGiven = NOT_GIVEN,
    search: str | None | NotGiven = NOT_GIVEN,
    ordering: str | None | NotGiven = NOT_GIVEN,
    limit: int | None | NotGiven = NOT_GIVEN,
    extra_query: dict[str, Any] | None = None,
    options: RequestOptions | None = None,
) -> SyncPager[models.ExperimentListV2]
client.experiments.list(params: ExperimentsListParams = {}, options?: RequestOptions): Pager<models.ExperimentListV2>

export interface ExperimentsListParams {
  created_at?: string | null;
  status?: string | null;
  dataset_id?: string | null;
  /** A search term. */
  search?: string | null;
  /** Which field to use when ordering the results. */
  ordering?: string | null;
  /** Number of results to return per page. */
  limit?: number | null;
}

Parameters

created_at string Optional
status string Optional
dataset_id string Optional
search string Optional

A search term.

ordering string Optional

Which field to use when ordering the results.

limit integer Optional

Number of results to return per page.

The pager sends page itself; pass limit to change the page size.

Returns

A pager of ExperimentListV2 items: iterate it, or call to_list() (Python) / toArray() (TypeScript).

Was this page helpful?

Questions & Discussion