Datasets & evals response models

The 42 response models returned by the datasets & evals resources of the Future AGI Python and TypeScript SDKs, with every field and its type.

The models returned by client.datasets, client.evals, client.experiments, client.optimizations, importable from fi.platform.models (Python) and @future-agi/sdk/platform/models (TypeScript). Types are language-neutral: string[] is a Python list[str] and a TypeScript string[], and a type named after another model links to that model. Parsing is tolerant by default: a response that leaves a field out or adds one never breaks a call. In Python, Client(strict_responses=True) rejects a response that misses a field the contract requires (see the overview); TypeScript has no strict mode.

Models

ModelFields
Columnid: string, name: string, data_type: string, dataset: string, source: string, source_id: string
CustomEvalConfigid: string, eval_template: string, name: string, config: any, mapping: any, project: string, filters: any, error_localizer: boolean, kb_id: string, model: string, eval_group: any
DatasetColumnsMutationResultmessage: string, data: Column[]
DatasetCreateStartedResultmessage: string, dataset_id: string, dataset_name: string, dataset_model_type: string
DatasetEvalStatsItemid: string, name: string, output_type: string, result: DatasetEvalStatsMetric[], total_pass_rate: number, total_avg: any, total_choices_avg: any, is_numeric_eval: boolean, is_numeric_eval_percentage: boolean
DatasetEvalStatsMetricid: string, name: string, total_cells: integer, output: any
DatasetListItemid: string, name: string, number_of_datapoints: integer, number_of_experiments: integer, number_of_optimisations: integer, derived_datasets: integer, created_at: string, dataset_type: string
DatasetNameItemdataset_id: string, name: string, model_type: string
DatasetNamesResultdatasets: DatasetNameItem[]
DatasetOptimizationColumnConfigItemid: string, name: string, is_visible: boolean
DatasetOptimizationDetailoptimiser_name: string, optimiser_type: string, model: string, model_deprecated: boolean, provider_logo: string, configuration: any, status: string, error_message: string, start_time: string, parameters: DatasetOptimizationParameterItem[], column_id: string, column_name: string, best_score: number, baseline_score: number, table: DatasetOptimizationTrialTableRow[], column_config: DatasetOptimizationColumnConfigItem[], optimizer_model_id: string, user_eval_templates: DatasetOptimizationEvalTemplateItem[]
DatasetOptimizationEvalTemplateItemid: string, eval_id: string, name: string, template_id: string
DatasetOptimizationParameterItemkey: string, label: string, description: string, value: any
DatasetOptimizationTrialEvalScorescore: number, percentage_change: number
DatasetOptimizationTrialTableRowid: string, trial: string, prompt: string, is_best: boolean, score_percentage_change: number, eval_scores: any
DatasetRunPromptStatsPromptid: string, name: string, input_token: number, output_token: number, total_token: number
DatasetRunPromptStatsResultavg_tokens: number, avg_cost: number, avg_time: number, prompts: DatasetRunPromptStatsPrompt[]
DatasetTableColumnid: string, name: string, data_type: string, is_visible: boolean, is_frozen: boolean, source_type: string, origin_type: string, source_id: string, order_index: integer, status: string, average_score: number, reason_column: boolean, is_numeric_eval: boolean, is_numeric_eval_percentage: boolean, eval_tag: string[], metadata: any, choices_map: any
DatasetTableMetadatadataset_name: string, experiment_id: string, experiment_name: string, total_rows: integer, total_pages: integer, page_size: integer, current_page_index: integer, has_more: boolean, next_page_index: integer, next_cursor: string, is_exact: boolean, snapshot_bound: boolean, error_messages: string[], status: any
DatasetTableResultmetadata: DatasetTableMetadata, column_config: DatasetTableColumn[], table: DatasetTableRow[], dataset_config: any, synthetic_dataset: boolean, synthetic_dataset_percentage: number, synthetic_regenerate: boolean, is_processing_data: boolean
DatasetTableRowrow_id: string
EvalExecutionResponseResultoutput: any, result: any, reason: string, score: any, metadata: any, log_id: string, error_localizer: any, error_details: any
EvalGroupid: string, name: string, organization: string, workspace: string, created_at: string, updated_at: string, description: string, created_by: any, is_sample: boolean
EvalListResultevals: any[], eval_recommendations: string[]
EvalTaskid: string, project: string, name: string, filters: object, sampling_rate: number, last_run: string, spans_limit: integer, run_type: string, row_type: string, status: string, start_time: string, end_time: string, created_at: string, updated_at: string, evals_details: any, evals: string[], failed_spans: any, progress: any
EvalTaskCreateResultid: string
EvalTaskMessageResultmessage: string
EvalTaskUpdateResultmessage: string, edit_type: string, task_id: string
EvalTemplateChartPointtimestamp: string, value: number
EvalTemplateCreateResponseResultid: string, name: string, version: string
EvalTemplateDetailResponseResultid: string, name: string, description: string, template_type: string, eval_type: string, instructions: string, model: string, output_type: string, pass_threshold: number, choice_scores: any, choices: any, multi_choice: boolean, code: string, code_language: string, required_keys: string[], owner: string, created_by_name: string, version_count: integer, current_version: string, tags: string[], check_internet: boolean, error_localizer_enabled: boolean, template_format: string, aggregation_enabled: boolean, aggregation_function: string, composite_child_axis: string, config: any, created_at: string, updated_at: string
EvalTemplateListItemid: string, name: string, template_type: string, eval_type: string, output_type: string, owner: string, created_by_name: string, version_count: integer, current_version: string, last_updated: string, thirty_day_chart: EvalTemplateChartPoint[], thirty_day_error_rate: EvalTemplateChartPoint[], thirty_day_run_count: integer, tags: string[]
EvalTemplateListResponseResultitems: EvalTemplateListItem[], total: integer, page: integer, page_size: integer
EvalTemplateUpdateResponseResultid: string, name: string, updated: boolean
EvalsConfigsCreateResultid: string
ExperimentDetailV2id: string, name: string, dataset_id: string, column_id: string, experiment_type: string, status: string, snapshot_dataset_id: string, prompt_configs: any, agent_configs: any, user_eval_metrics: any, created_at: string
ExperimentListV2id: string, name: string, status: string, experiment_type: string, eval_templates_count: any, created_at: string, models_count: any, agents_count: any, dataset: string
LocalFileDatasetCreateStartedResultmessage: string, dataset_id: string, dataset_name: string, dataset_model_type: string, processing_status: string, estimated_rows: integer, estimated_columns: integer
ModelHubSuccessMessageResultsuccess: string
OptimizationsCreateResultid: string, name: string, optimizer_algorithm: string, optimizer_config: any, created_at: string
_EvalTaskListResultResponsemetadata: any, table: _EvalTaskListRowResponse[], config: any[]
_EvalTaskListRowResponseid: string, name: string, project_name: string, status: string, run_type: string, filters_applied: any, created_at: string, evals_applied: string[], sampling_rate: number, last_run: string
Was this page helpful?

Questions & Discussion