R-ORD GOLD-ONLY arbitrary-cut
formula_1 · dev_20251106-00000-of-00001 from https://huggingface.co/datasets/birdsql/bird_sql_dev_20251106/resolve/3c11fb193e5439b338e23677fa0aae11e8b85db9/data/dev_20251106-00000-of-00001.json (commit 3c11fb19, downloaded 2026-09-07)
Which was Lewis Hamilton first race? What was his points recorded for his first race event?
the hint the set supplies: first race refers to min(Year); Lewis Hamiltonrefers to the full name of the driver; Full name of the driver refers to drivers.forename and drivers.surname;
This question was audited without a prediction beside it, so there is nothing to compare the gold with. The probes below read the gold alone.
read by hand, 2026-09-07: wrong the gold does not answer its question on this data
the gold orders by year and cuts at one, and 17 races share the first year, so which of that season's races comes back is arbitrary
a maintainer's reading of this question, out of classification.json, copied from plans/reports/bird-dev-sqlite-260907/classification.json. It is not a verdict and nothing above it was computed from it.
SELECT T1.name, T2.points FROM races AS T1 INNER JOIN driverStandings AS T2 ON T2.raceId = T1.raceId INNER JOIN drivers AS T3 ON T3.driverId = T2.driverId WHERE T3.forename = 'Lewis' AND T3.surname = 'Hamilton' ORDER BY T1.year ASC LIMIT 1sha256:972af0b8ff1f4947ddd04a73f352f7e5ca4d34807b0d5e9d32c2bb7fa3fd63f6
from evidence-gold.json, 1 row
| nameTEXT | pointsREAL |
|---|---|
| Australian Grand Prix | 6.0 |
A smell is a mechanical reason to read this gold statement again. It is a heuristic: it does not state that the statement is wrong, and a maintainer decides.
this statement orders by a text column holding only numbers, and ordering it as a number gives a different answer, so the gold may be sorting 9.5 above 10
no ORDER BY key resolves to a text column
{
"heuristic": true,
"reason": "no ORDER BY key resolves to a text column",
"keys": [
{
"key": "T1.year",
"column": "races.year",
"declared_type": "INTEGER",
"not_applicable": "the column is not declared as text"
}
]
}this statement cuts its result at a LIMIT that does not decide which rows come back, so a different but equally correct statement can return other rows and score zero
from smells.json, 10 rows
| Australian Grand Prix | 6.0 | 2007 |
| Malaysian Grand Prix | 14.0 | 2007 |
| Bahrain Grand Prix | 22.0 | 2007 |
| Spanish Grand Prix | 30.0 | 2007 |
| Monaco Grand Prix | 38.0 | 2007 |
| Canadian Grand Prix | 48.0 | 2007 |
| United States Grand Prix | 58.0 | 2007 |
| French Grand Prix | 64.0 | 2007 |
| British Grand Prix | 70.0 | 2007 |
| European Grand Prix | 70.0 | 2007 |
{
"heuristic": true,
"cut": 1,
"offset": 0,
"distinct_kept": false,
"unbounded_sql": "SELECT T1.name, T2.points, T1.year AS attestql_ordering_key_0 FROM races AS T1 INNER JOIN driverStandings AS T2 ON T2.raceId = T1.raceId INNER JOIN drivers AS T3 ON T3.driverId = T2.driverId WHERE T3.forename = 'Lewis' AND T3.surname = 'Hamilton' ORDER BY T1.year ASC",
"unbounded_rows": 201,
"projected_columns": [
"name",
"points"
],
"ordering_key_columns": [
"attestql_ordering_key_0"
],
"ordering_keys": [
{
"key": "T1.year",
"direction": "asc",
"nulls": "first",
"nulls_first_in_effect": true,
"returned_rows_null_in_this_key": 0,
"fires": false
}
],
"tied_at_the_cut": {
"positions": [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9
],
"tied_rows": 17,
"distinct_projected_answers": 17,
"rows": [
[
{
"type": "str",
"value": "Australian Grand Prix"
},
{
"type": "dec",
"value": "6.0"
}
],
[
{
"type": "str",
"value": "Malaysian Grand Prix"
},
{
"type": "dec",
"value": "14.0"
}
],
[
{
"type": "str",
"value": "Bahrain Grand Prix"
},
{
"type": "dec",
"value": "22.0"
}
],
[
{
"type": "str",
"value": "Spanish Grand Prix"
},
{
"type": "dec",
"value": "30.0"
}
],
[
{
"type": "str",
"value": "Monaco Grand Prix"
},
{
"type": "dec",
"value": "38.0"
}
],
[
{
"type": "str",
"value": "Canadian Grand Prix"
},
{
"type": "dec",
"value": "48.0"
}
],
[
{
"type": "str",
"value": "United States Grand Prix"
},
{
"type": "dec",
"value": "58.0"
}
],
[
{
"type": "str",
"value": "French Grand Prix"
},
{
"type": "dec",
"value": "64.0"
}
],
[
{
"type": "str",
"value": "British Grand Prix"
},
{
"type": "dec",
"value": "70.0"
}
],
[
{
"type": "str",
"value": "European Grand Prix"
},
{
"type": "dec",
"value": "70.0"
}
]
]
},
"case": "tie-at-the-cut"
}rerun over the same rows in another physical order this statement gives another answer, so its result depends on how the rows are stored and not only on the data
{
"heuristic": true,
"rule": "R-ORD",
"baseline_result_hash": "sha256:972af0b8ff1f4947ddd04a73f352f7e5ca4d34807b0d5e9d32c2bb7fa3fd63f6",
"baseline_result": {
"columns": [
{
"name": "name",
"declared_type": "TEXT"
},
{
"name": "points",
"declared_type": "REAL"
}
],
"row_count": 1,
"truncated": false,
"rows_shown": 1,
"rows": [
[
{
"type": "str",
"value": "Australian Grand Prix"
},
{
"type": "dec",
"value": "6.0"
}
]
],
"result_hash": "sha256:972af0b8ff1f4947ddd04a73f352f7e5ca4d34807b0d5e9d32c2bb7fa3fd63f6"
},
"planner_statistics": {},
"shuffle": {
"seed": "1",
"row_limit": 300000,
"tables": [
"races",
"driverStandings",
"drivers"
],
"tables_not_shuffled": [],
"tables_skipped_for_size": {
"lapTimes": 420369
},
"tables_not_reached_by_a_copy": {}
},
"shuffled_copies": {
"run": true,
"verdict": "equal",
"differs": false,
"result_hash": "sha256:972af0b8ff1f4947ddd04a73f352f7e5ca4d34807b0d5e9d32c2bb7fa3fd63f6",
"result": {
"columns": [
{
"name": "name",
"declared_type": "TEXT"
},
{
"name": "points",
"declared_type": "REAL"
}
],
"row_count": 1,
"truncated": false,
"rows_shown": 1,
"rows": [
[
{
"type": "str",
"value": "Australian Grand Prix"
},
{
"type": "dec",
"value": "6.0"
}
]
],
"result_hash": "sha256:972af0b8ff1f4947ddd04a73f352f7e5ca4d34807b0d5e9d32c2bb7fa3fd63f6"
}
},
"plan_variant": {
"run": false,
"reason": "the plan variant was not asked for"
}
}SELECT T1.name, T2.points FROM races AS T1 INNER JOIN driverStandings AS T2 ON T2.raceId = T1.raceId INNER JOIN drivers AS T3 ON T3.driverId = T2.driverId WHERE T3.forename = 'Lewis' AND T3.surname = 'Hamilton' ORDER BY T1.year ASC LIMIT 1result_hash sha256:972af0b8ff1f4947ddd04a73f352f7e5ca4d34807b0d5e9d32c2bb7fa3fd63f6 recomputed from this JSON: match
record_hash sha256:559e282ad98596315598975d2ff62f46a628142ba13fa0d9dc6c9f95b5f077b0 recomputed from this JSON: match
from evidence-gold.json, 1 row
| nameTEXT | pointsREAL |
|---|---|
| Australian Grand Prix | 6.0 |
re-run this statement read-only against SQLite 3.53.4 | file=/private/tmp/attestql-runs/data/dev/dev_databases/formula_1/formula_1.sqlite | size=22360064 under the session settings and over the data this record's fixture digest names, and compare the two results under R-ORD