Random Search

When to use Random Search

Random Search is the cheapest way to find out how much headroom a prompt has before reaching for a directed optimizer, one that uses each round’s scores to steer the next (see choosing an optimizer to compare it against the other five). A run returns the highest-scoring variation it found along with its score. It generates a fixed batch of independent variations of your starting prompt and scores each one against your dataset: no variation feeds into the next, so the score for variation 2 has no effect on what variation 3 looks like. Run it from the platform UI or the Python SDK.

Parameters

The On-screen label column is the field name shown when you run this optimizer from the UI; see Run an optimization for the full form walkthrough. The Default column shows what applies when you don’t set the value yourself: the UI form’s prefilled value, or the SDK’s fallback when the argument is omitted.

More variations cover more of the prompt space but cost proportionally more generation and evaluation calls, since each one is scored independently. Start at 3 for a quick read on headroom.

The table above covers only this optimizer’s tuning knob. evaluator, data_mapper, and dataset, also passed in the example below, are shared by every optimizer’s optimize() call and are covered in the SDK reference.

Usage

Requires pip install agent-opt, which provides the fi.opt modules imported below, plus an FI_API_KEY and FI_SECRET_KEY pair (see API keys for where to get them). Pass them as environment variables or, as below, directly into Evaluator.

from fi.opt.optimizers import RandomSearchOptimizer
from fi.opt.generators import LiteLLMGenerator
from fi.opt.datamappers import BasicDataMapper
from fi.opt.base.evaluator import Evaluator

# Generator holding the starting prompt
generator = LiteLLMGenerator(
    model="gpt-4o-mini",
    prompt_template="Summarize this article: {article}"
)

# Evaluator that scores each variation
evaluator = Evaluator(
    eval_template="summary_quality",
    eval_model_name="turing_flash",
    fi_api_key="your_key",
    fi_secret_key="your_secret"
)

# Maps generator output and dataset fields to what the evaluator expects.
# "generated_output" is the generator's fixed output key; "article" is
# the dataset field from this example and should match your own data.
data_mapper = BasicDataMapper(
    key_map={"input": "article", "output": "generated_output"}
)

# Dataset: a plain list of dicts, one per example the optimizer scores the prompt against
my_dataset = [
    {"article": "The James Webb Space Telescope has captured its clearest images yet of a distant exoplanet's atmosphere, revealing traces of carbon dioxide and methane."},
    {"article": "Researchers have discovered a new enzyme that breaks down PET plastic at room temperature, far faster than any previously known enzyme."},
]

optimizer = RandomSearchOptimizer(
    generator=generator,
    num_variations=3
)

result = optimizer.optimize(
    evaluator=evaluator,
    data_mapper=data_mapper,
    dataset=my_dataset
)

print(f"Final score: {result.final_score:.4f}")
print(f"Best prompt:\n{result.best_generator.get_prompt_template()}")

result.final_score is the winning variation’s score and result.best_generator.get_prompt_template() is its prompt text; see reading the result for what these fields mean and how to use them.

summary_quality and turing_flash are real built-in names; see eval templates and evaluator models for the full lists.

A successful run prints something like:

Final score: 0.8532
Best prompt:
Summarize this article in 2-3 sentences, covering the main finding and its significance.

Keep exploring