API Call
Assesses the validity and correctness of responses from external APIs, ensuring that they align with expected criteria. This evaluation is essential for systems that depend on API integrations, helping to verify response structure, data accuracy, and reliability.
Evaluation Using Interface
Input:
- Required Inputs:
- response: The column containing the API’s response content (e.g., JSON body, status code).
- Optional Inputs:
- None specified for this evaluation.
- Configuration Parameters:
- (Optional) expected_status_code: Integer - The expected HTTP status code for a successful call (e.g., 200).
- (Optional) validate_json_body: Boolean - Whether to check if the response body is valid JSON.
Output:
- Result: Passed / Failed
Interpretation:
- Passed: Indicates that the API call response met the validation criteria (e.g., matched the
expected_status_code
, contained valid JSON ifvalidate_json_body
was true). - Failed: Suggests an issue with the API response based on the configured criteria (e.g., unexpected status code, malformed JSON body).
Evaluation Using Python SDK
Click here to learn how to setup evaluation using the Python SDK.
Input Type | Parameter | Type | Description |
---|---|---|---|
Required Inputs | response | string | The API response content (e.g., JSON body as a string, or status code). |
Configuration Parameters | expected_status_code | int | (Optional) The expected HTTP status code for success. |
validate_json_body | bool | (Optional) If true, checks if the response string is valid JSON. Default: False . |
Output | Type | Description |
---|---|---|
Result | bool | Returns 1.0 if the validation passes, 0.0 otherwise (Fail). |
What to do when API Call Evaluation Fails
Check the API endpoint and parameters to ensure they are correctly configured. Reviewing the response for error messages or status codes can help identify the cause of failure.
Differentiating API Call Eval with Function Calling Eval
The API Call evaluation focuses on making network requests to external services and validating the responses, while Evaluate LLM Function Calling examines whether LLMs correctly identify and execute function calls.
API calls are used for external interactions like retrieving data or triggering actions, while function call evaluation ensures that LLMs correctly interpret and execute function calls based on input prompts.
They differ in validation criteria, where API calls are assessed based on response content, status codes, and data integrity, the function call evaluation focuses on the accuracy of function call identification and parameter extraction.
Was this page helpful?