Skip to content

SQL Syntax Match


SQL Syntax Match evaluates the syntactic equivalence between generated SQL queries and a set of ground truth queries. The strict comparison can tolerate formatting changes.

Introduced in version 0.3.9.

Example Usage

Required data items: answer, ground_truth_answers

from continuous_eval.metrics.code import SQLSyntaxMatch
sql_syntax_match = SQLSyntaxMatch()
datum = {
"answer": "SELECT * FROM users;"",
"ground_truth_answers": [
"SELECT * from users;"
metric = SQLSyntaxMatch()

You can optionally initialize the metric to use optimized SQL queries using the sqlglot optimizer and optionally pass in the schema. For example:

schema={"x": {"A": "INT", "B": "INT", "C": "INT", "D": "INT", "Z": "STRING"}}
sql_syntax_match_optimized = SQLSyntaxSimilarity(optimized=True, schema=schema)

Example Output

"SQL_Syntax_Match": 1.0