Create a dataset
Six ways to create a dataset and get data into it.
A dataset starts as just a name in your organization; everything else comes from how you fill it. Future AGI gives you six ways to do that, each landing you on the same dataset table of rows and columns you can keep editing afterward.
Every method starts the same way: from the dataset list (Dataset in the left nav), click Add Dataset. That opens a panel with six tiles:
| Method | Reach for it when |
|---|---|
| Add data using SDK | You’re scripting the setup or pushing rows from your own pipeline |
| Upload a file (JSON, CSV) | You already have test cases in a CSV, Excel, or JSON export |
| Create Synthetic Data | You don’t have real data yet, but know the shape of the examples you need |
| Add datasets Manually | You’re hand-building a small set and want an empty grid to fill in |
| Import from HuggingFace | The data you need already exists as a Hugging Face dataset |
| Add from existing model dataset or experiment | You want to branch off a dataset or experiment you already have |
Note
Add data using the SDK
For scripting dataset creation, or when you’d rather write rows in code than click through the grid.
In the Add dataset panel, pick Add data using SDK and name the dataset. Future AGI creates an empty dataset and drops you on its Data tab, which shows the dataset’s name, ID, API key, and secret key alongside a ready-to-run code snippet. Copy the snippet below and run it against your dataset to add columns and rows.
An empty dataset starts with at most 10 rows; add the rest with the SDK.
# pip install futureagi
import os
from fi.datasets import Dataset
from fi.datasets.types import Cell, Column, DataTypeChoices, Row, SourceChoices
os.environ["FI_API_KEY"] = "<fi_api_key>"
os.environ["FI_SECRET_KEY"] = "<fi_secret_key>"
# Get the dataset you just created
dataset = Dataset.get_dataset_config("support-agent-eval")
# Define columns
columns = [
Column(name="user_query", data_type=DataTypeChoices.TEXT, source=SourceChoices.OTHERS),
Column(name="response_quality", data_type=DataTypeChoices.INTEGER, source=SourceChoices.OTHERS),
Column(name="is_helpful", data_type=DataTypeChoices.BOOLEAN, source=SourceChoices.OTHERS),
]
# Define rows
rows = [
Row(order=1, cells=[
Cell(column_name="user_query", value="What is machine learning?"),
Cell(column_name="response_quality", value=8),
Cell(column_name="is_helpful", value=True),
]),
Row(order=2, cells=[
Cell(column_name="user_query", value="Explain quantum computing"),
Cell(column_name="response_quality", value=9),
Cell(column_name="is_helpful", value=True),
]),
]
dataset = dataset.add_columns(columns=columns)
dataset = dataset.add_rows(rows=rows)import { Dataset, DataTypeChoices, createRow, createCell } from "@future-agi/sdk";
process.env["FI_API_KEY"] = "<fi_api_key>";
process.env["FI_SECRET_KEY"] = "<fi_secret_key>";
async function main() {
// Get the dataset you just created
const dataset = await Dataset.open("support-agent-eval");
// Define columns
const columns = [
{ name: "user_query", dataType: DataTypeChoices.TEXT },
{ name: "response_quality", dataType: DataTypeChoices.INTEGER },
{ name: "is_helpful", dataType: DataTypeChoices.BOOLEAN },
];
// Define rows
const rows = [
createRow({
cells: [
createCell({ columnName: "user_query", value: "What is machine learning?" }),
createCell({ columnName: "response_quality", value: 8 }),
createCell({ columnName: "is_helpful", value: true }),
],
}),
createRow({
cells: [
createCell({ columnName: "user_query", value: "Explain quantum computing" }),
createCell({ columnName: "response_quality", value: 9 }),
createCell({ columnName: "is_helpful", value: true }),
],
}),
];
await dataset.addColumns(columns);
await dataset.addRows(rows);
}
main();curl --request POST \
--url https://api.futureagi.com/model-hub/develops/<dataset_id>/add_columns/ \
--header 'X-Api-Key: <fi_api_key>' \
--header 'X-Secret-Key: <fi_secret_key>' \
--header 'content-type: application/json' \
--data '{
"new_columns_data": [
{"name": "user_query", "data_type": "text"},
{"name": "response_quality", "data_type": "integer"},
{"name": "is_helpful", "data_type": "boolean"}
]
}'See the Datasets SDK reference for the full Dataset API.
Upload a file
For bringing in test cases you already have as a file, instead of typing them in.
In the Add dataset panel, pick Upload a file (JSON, CSV) and name the dataset. Drop or browse to your file: accepted formats are .csv, .xls, .xlsx, .json, and .jsonl, up to 25 MB. The dataset appears on your list right away. Future AGI processes the file in the background with a visible progress state until it’s done.
Create synthetic data
For when you don’t have real data yet, but know the shape of the examples you need.
In the Add dataset panel, pick Create Synthetic Data and name the dataset. From there, Future AGI walks you through describing the schema and generates rows for you. See Synthetic Data if you want to regenerate later.
Add a dataset manually
For hand-building a small dataset from scratch when you already know its shape.
In the Add dataset panel, pick Add datasets Manually, name the dataset, and choose how many rows and how many columns to start with, up to 100 of each. Future AGI creates the dataset with that many empty rows and columns, ready for you to fill in.
Import from Hugging Face
For pulling in a Hugging Face dataset instead of typing test cases by hand.
In the Add dataset panel, pick Import from HuggingFace and paste the Hugging Face dataset ID. Click Load Dataset, then pick the Subset and Split you want. Name the new dataset to finish. Only the first 100 rows of the source are ingested.
Add from an existing dataset or experiment
For branching off a dataset or experiment you already have.
In the Add dataset panel, pick Add from existing model dataset or experiment and choose the dataset or experiment you want to copy from. Choose whether to bring over Import Data or Import data and prompt configuration, then select which columns to include. Name the new dataset to finish.
Dive deeper
Questions & Discussion