Power Of Combining Future AGI And Portkey
.env
File: Create a .env
file in your project root to securely store your Portkey API Key.
dataclasses
to create structured objects for our model configurations and test results, which makes the code cleaner and more maintainable.
setup_tracing
method configures FutureAGI.
register()
: Initializes a tracing project. We give it a project_name
and a project_version_name
to organize our experiments.eval_tags
: This is where the magic happens. We define a list of EvalTag
objects that tell FutureAGI what to evaluate.EvalTag
:
type
& value
: Specifies that this evaluation should run on every LLM call span.eval_name
: The built-in evaluation to use (e.g., CONTEXT_ADHERENCE
).custom_eval_name
: A user-friendly name that will appear in the FutureAGI dashboard (e.g., “Response_Quality”).mapping
: This is crucial. It tells the evaluator where to find the necessary data within the trace. Here, we map the LLM’s input prompt to the context
parameter of the evaluator and the LLM’s response to the output
parameter.PortkeyInstrumentor().instrument()
: This line activates the instrumentation, linking our FutureAGI setup to any Portkey client created afterward.test_model
function orchestrates a single test run.
Portkey
client using the model-specific Virtual Key.client.chat.completions.create()
. Because of our instrumentation in Step 2, this call is automatically traced.TestResult
object.