Evaluation Using Interface
Input:- input: The input column provided to the LLM that triggers the function call.
- output: Column which has the resulting function call or response generated by the LLM.
- Result: Passed / Failed
- Passed: The LLM correctly identified that a function/tool call was necessary based on the input and accurately extracted the required parameters into the expected format.
- Failed: The LLM did not correctly handle the function call requirement. This could mean it either failed to recognize the need for a function call altogether, or it recognized the need but extracted incorrect or incomplete parameters from the input.
Evaluation Using SDK
Click here to learn how to setup evaluation using SDK.
Input | Parameter | Type | Description |
---|---|---|---|
Required | input | string | input text provided to the LLM that triggers the function call. |
output | string | output text which has the resulting function call or response generated by the LLM. |
Output | Type | Description |
---|---|---|
Result | bool | Returns 0 or 1 . 0 : The LLM did not correctly handle the function call requirement. 1 : The LLM correctly identified that a function/tool call was necessary. |