Text to SQL

Text to SQL checks whether a generated SQL query correctly represents a natural language request. Run it wherever a model translates user intent into a database query.

What it does

Text to SQL is an LLM-as-Judge eval. It reads the natural language query and the generated SQL, then checks whether the SQL correctly represents the request.

Input

Output

Run it from code

Call evaluate() with the template name and the eval’s required inputs. It returns the score and the reason.

from fi.evals import evaluate

result = evaluate(
    "text_to_sql",
    input="List the names of all employees who work in the sales department.",
    output="SELECT name FROM employees WHERE department = 'sales';",
    model="turing_flash",
)

print(result.score)
print(result.reason)

import { evaluate } from "@future-agi/ai-evaluation";

const result = await evaluate(
  "text_to_sql",
  {
    input: "List the names of all employees who work in the sales department.",
    output: "SELECT name FROM employees WHERE department = 'sales';"
  },
  { modelName: "turing_flash" }
);

console.log(result);

When to use

Run Text to SQL wherever a natural language interface generates queries against a database.

• Text outputs from natural-language-to-SQL agents or copilots
• Analytics tools where users ask questions in plain language and expect a correct query back
• Regression checks after changing a schema or prompt that drives SQL generation

What to do when Text to SQL fails

If the SQL query is evaluated as incorrect, ensure the SQL syntax is correct and follows standard conventions, and verify that all tables and columns referenced match the database schema implied by the natural language query. Check that the query filters for exactly the data requested, no more and no less, and that appropriate joins are used when multiple tables are involved.

Confirm the query handles potential edge cases like NULL values appropriately, and use the correct data types for values in comparisons (for example, quotation marks for strings). For complex queries, consider breaking them down into simpler parts for troubleshooting.