Hugging Face Dataset Import

Use HuggingfaceDatasetConfig to specify which dataset, subset, split, and how many rows to pull, then pass it as the source argument to dataset.create().

This example imports 50 rows from the SmolLM-Corpus cosmopedia-v2 subset: synthetic textbook-style content with prompts, generated text, audience labels, and format tags.

import os
from fi.datasets import Dataset, DatasetConfig, HuggingfaceDatasetConfig
from fi.utils.types import ModelTypes

hf_config = HuggingfaceDatasetConfig(
    name="HuggingFaceTB/smollm-corpus",
    subset="cosmopedia-v2",
    split="train",
    num_rows=50,
)

dataset = Dataset(
    dataset_config=DatasetConfig(
        name="smollm-cosmopedia-import",
        model_type=ModelTypes.GENERATIVE_LLM,
    ),
    fi_api_key=os.environ["FI_API_KEY"],
    fi_secret_key=os.environ["FI_SECRET_KEY"],
)

dataset = dataset.create(source=hf_config)

print(f"Dataset created: {dataset.dataset_config.name}")
print(f"Dataset ID: {dataset.dataset_config.id}")

You should see:

Dataset created: smollm-cosmopedia-import
Dataset ID: a1b2c3d4-...

Open Dataset in the left sidebar. Your new dataset appears in the list. Click it to browse the imported rows and columns.

The cosmopedia-v2 subset includes columns like prompt, text, audience, format, and token_length, ready for evaluation.

The prompt column holds the generation instruction and text holds the generated output, a natural fit for a completeness evaluation that checks whether the output fully addresses the input.

required_keys_to_column_names must map to columns that actually exist in the dataset. Mapping output to a column this dataset doesn’t have raises a DatasetError:

dataset.add_evaluation(
    name="completeness-check",
    eval_template="completeness",
    required_keys_to_column_names={
        "input": "prompt",
        "output": "generated_text",
    },
    model="turing_small",
    run=True,
    reason_column=True,
)

You should see:

DatasetError: Column 'generated_text' (mapped from key 'output') not found in dataset 'smollm-cosmopedia-import'.

The cosmopedia-v2 subset names its output column text, not generated_text. Fix the mapping and rerun:

dataset = dataset.add_evaluation(
    name="completeness-check",
    eval_template="completeness",
    required_keys_to_column_names={
        "input": "prompt",
        "output": "text",
    },
    model="turing_small",
    run=True,
    reason_column=True,
)

print("Evaluation 'completeness-check' started")

You should see:

Evaluation 'completeness-check' started

You should see the completeness-check column fill in on the dashboard, scoring each row on whether its text fully addresses its prompt.

add_evaluation(run=True) starts scoring and returns immediately, it does not wait for the run to finish. Confirm the evaluation has completed before downloading, either by checking that the completeness-check column is filled in on the dashboard, or with get_eval_stats():

stats = dataset.get_eval_stats()
print(stats)

Once the stats show the run complete, pull the evaluated dataset back as a CSV or a pandas DataFrame.

As CSV:

dataset.download(file_path="smollm_scored.csv")
print("Downloaded scored results to smollm_scored.csv")

As a pandas DataFrame:

df = dataset.download(load_to_pandas=True)

# Print all column names to see the exact eval and reason column names
print("Columns:", list(df.columns))
print(df[["completeness-check", "completeness-check_reason"]].head())

You should see the eval column (named after the evaluation, completeness-check) and its matching _reason column, for example:

Columns: ['prompt', 'text', 'token_length', 'audience', 'format', 'seed_data', 'completeness-check', 'completeness-check_reason']
  completeness-check                          completeness-check_reason
0              Passed  The generated text fully covers the prompt's...
1              Failed  The output omits the audience-specific frami...

dataset.delete()
print("Dataset deleted")