Input | |||
---|---|---|---|
Required Input | Type | Description | |
input | string | The natural language query or instruction. | |
output | string | The generated SQL query. |
Output | ||
---|---|---|
Field | Description | |
Result | Returns Passed if the SQL query correctly represents the natural language request, or Failed if it doesn’t. | |
Reason | Provides a detailed explanation of why the SQL query was classified as correct or incorrect. |
What to do If you get Undesired Results
If the SQL query is evaluated as incorrect (Failed) and you want to improve it:- Ensure the SQL syntax is correct and follows standard conventions
- Verify that all tables and columns referenced match the database schema implied by the natural language query
- Check that the query filters for exactly the data requested (no more, no less)
- Make sure appropriate joins are used when multiple tables are involved
- Confirm that the query handles potential edge cases like NULL values appropriately
- Use the correct data types for values in comparisons (e.g., quotation marks for strings)
- For complex queries, consider breaking them down into simpler parts for troubleshooting
Comparing Text to SQL with Similar Evals
- Task Completion: While Text to SQL focuses specifically on converting natural language to SQL queries, Task Completion evaluates whether a response completes the requested task more generally.
- Evaluate Function Calling: Text to SQL evaluates SQL generation specifically, whereas Evaluate Function Calling assesses the correctness of function calls and parameters more broadly.
- Is Code: Text to SQL evaluates the correctness of SQL generation, while Is Code detects whether content contains code of any type.