gold
SELECT DISTINCT team_fifa_api_id FROM Team_Attributes WHERE buildUpPlaySpeed > 50 AND buildUpPlaySpeed < 60this statement states no ordering of its own
sha256:1f7a83843d78e06ee35755d20fc3ce45b6c89aaf7eac10e855f720532e328e6b
The question
Give the team_fifa_api_id of teams with more than 50 but less than 60 build-up play speed.
the hint the set supplies: teams with more than 50 but less than 60 build-up play speed refers to buildUpPlaySpeed >50 AND buildUpPlaySpeed <60;
NOT_EQUAL states that these two statements disagree on this data under this rule. It does not state which of them is wrong.
read by hand, 2026-09-04: A a wrong answer the benchmark credited: another table or projection; a scalar repeated per row of an unrelated table; a list at least twice the answer's length where the question asks for the things, not the rows
gold DISTINCT over 161 ids; the prediction returns 356, so the count of teams is misstated
a maintainer's reading of this question, out of classification.json, copied from plans/reports/prediction-mode-260904-real-predictions/classification.json. It is not a verdict and nothing above it was computed from it.
The statements
second
SELECT team_fifa_api_id
FROM Team_Attributes
WHERE buildUpPlaySpeed > 50 AND buildUpPlaySpeed < 60this statement states no ordering of its own
sha256:1d5fb5460cba8d3b98439e43146df64c792a0e1c0f82b8cf5e922b2b48ba88c2
The marked tokens are where the two texts differ. Two statements that differ everywhere can return the same rows, and two that differ in one token can return other rows; the verdict above is read off the results.
The rows they differ in
in gold, not in the prediction, 0 rows
from counterexample.json, 0 rows, up to 25 shown per side
| side | team_fifa_api_idint8 |
|---|---|
| no rows |
in the prediction, not in gold, 195 rows
from counterexample.json, 195 rows, up to 25 shown per side, 25 on this page
| side | times | team_fifa_api_idint8 |
|---|---|---|
| second | 4 | 110569 |
| second | 4 | 13 |
| second | 4 | 15005 |
| second | 4 | 1715 |
| second | 4 | 242 |
| second | 4 | 31 |
| second | 4 | 4 |
| second | 4 | 462 |
| second | 4 | 900 |
| second | 3 | 100879 |
| second | 3 | 110329 |
| second | 3 | 110724 |
| second | 3 | 110832 |
| second | 3 | 111239 |
| second | 3 | 15 |
| second | 3 | 165 |
| second | 3 | 1832 |
| second | 3 | 203 |
| second | 3 | 229 |
| second | 3 | 232 |
| second | 3 | 237 |
| second | 3 | 449 |
| second | 3 | 450 |
| second | 3 | 459 |
| second | 3 | 468 |
What the benchmark would have said
BIRD's own check: 1
set(second_rows) == set(gold_rows), float4/float8 cells as Python float as psycopg2 returns them, numeric as Decimal
https://github.com/bird-bench/mini_dev/blob/main/evaluation/evaluation_ex.py
- gold_rows
- 161
- second_rows
- 356
- gold_distinct_rows
- 161
- second_distinct_rows
- 161
the test-suite check: 0
result_eq: equal row counts and equal column counts, each row unordered as a quick rejection, then the two equal as a list when the gold text holds ORDER BY and as a multiset otherwise, under some permutation of the columns; DISTINCT is not stripped and re-executed, and the cells are PostgreSQL's as psycopg2 returns them
ruiqi-zhong/test-suite-sql-eval, exec_eval.py, result_eq, at commit 48cb78ec: https://github.com/ruiqi-zhong/test-suite-sql-eval/blob/48cb78ecf7f610620206283846c76751b18a1326/exec_eval.py
- order_matters
- false
- gold_rows
- 161
- second_rows
- 356
- gold_columns
- 1
- second_columns
- 1
the class: multiplicity
The class states what makes these two results unequal under this rule, read off the two results and nothing else. It does not state which of the two statements is wrong.
- gold_types
- ["int8"]
- second_types
- ["int8"]
- multiset_equal
- false
- set_equal
- true
- order_equal
- false
- shorter_result_is_a_prefix
- false
The results
gold, 161 rows
from counterexample.json, 161 rows, 25 on this page
| team_fifa_api_idint8 |
|---|
| 485 |
| 110744 |
| 477 |
| 71 |
| 1889 |
| 68 |
| 229 |
| 481 |
| 52 |
| 70 |
| 80 |
| 1971 |
| 112225 |
| 1861 |
| 378 |
| 874 |
| 450 |
| 10030 |
| 1799 |
| 900 |
| 110745 |
| 1750 |
| 462 |
| 237 |
| 69 |
second, 356 rows
from counterexample.json, 356 rows, 25 on this page
| team_fifa_api_idint8 |
|---|
| 434 |
| 77 |
| 77 |
| 77 |
| 614 |
| 614 |
| 614 |
| 614 |
| 1901 |
| 1901 |
| 650 |
| 650 |
| 650 |
| 1861 |
| 229 |
| 229 |
| 229 |
| 229 |
| 111989 |
| 1 |
| 1 |
| 240 |
| 240 |
| 100409 |
| 100409 |
The probes
A smell is a mechanical reason to read this gold statement again. It is a heuristic: it does not state that the statement is wrong, and a maintainer decides.
ordering-over-numeric-text not applicable
this statement orders by a text column holding only numbers, and ordering it as a number gives a different answer, so the gold may be sorting 9.5 above 10
the statement states no top level ORDER BY
what it measured
{
"heuristic": true,
"reason": "the statement states no top level ORDER BY"
}arbitrary-cut not applicable
this statement cuts its result at a LIMIT that does not decide which rows come back, so a different but equally correct statement can return other rows and score zero
the statement states no LIMIT
what it measured
{
"heuristic": true,
"reason": "the statement states no LIMIT"
}not-a-function-of-the-data quiet
rerun over the same rows in another physical order this statement gives another answer, so its result depends on how the rows are stored and not only on the data
what it measured
{
"heuristic": true,
"rule": "R-SET",
"baseline_result_hash": "sha256:1f7a83843d78e06ee35755d20fc3ce45b6c89aaf7eac10e855f720532e328e6b",
"baseline_result": {
"columns": [
{
"name": "team_fifa_api_id",
"declared_type": "int8"
}
],
"row_count": 161,
"truncated": false,
"rows_shown": 10,
"rows": [
[
{
"type": "int",
"value": 485
}
],
[
{
"type": "int",
"value": 110744
}
],
[
{
"type": "int",
"value": 477
}
],
[
{
"type": "int",
"value": 71
}
],
[
{
"type": "int",
"value": 1889
}
],
[
{
"type": "int",
"value": 68
}
],
[
{
"type": "int",
"value": 229
}
],
[
{
"type": "int",
"value": 481
}
],
[
{
"type": "int",
"value": 52
}
],
[
{
"type": "int",
"value": 70
}
]
],
"result_hash": "sha256:1f7a83843d78e06ee35755d20fc3ce45b6c89aaf7eac10e855f720532e328e6b"
},
"planner_statistics": {
"team_attributes": {
"last_analyze": null,
"last_autoanalyze": "2026-09-08 05:19:47.099723+00",
"n_mod_since_analyze": 0
}
},
"shuffle": {
"seed": "1",
"row_limit": 300000,
"tables": [
"team_attributes"
],
"tables_not_shuffled": [],
"tables_skipped_for_size": {
"laptimes": 400524,
"legalities": 427907,
"posthistory": 303155,
"trans": 1056320,
"yearmonth": 383282
},
"tables_not_reached_by_a_copy": {}
},
"shuffled_copies": {
"run": true,
"verdict": "equal",
"differs": false,
"result_hash": "sha256:c7fede3477f21b2f77b749c41badaa7465b6edd31397c900e37f5f6da611de83",
"result": {
"columns": [
{
"name": "team_fifa_api_id",
"declared_type": "int8"
}
],
"row_count": 161,
"truncated": false,
"rows_shown": 10,
"rows": [
[
{
"type": "int",
"value": 1
}
],
[
{
"type": "int",
"value": 3
}
],
[
{
"type": "int",
"value": 4
}
],
[
{
"type": "int",
"value": 7
}
],
[
{
"type": "int",
"value": 10
}
],
[
{
"type": "int",
"value": 13
}
],
[
{
"type": "int",
"value": 15
}
],
[
{
"type": "int",
"value": 17
}
],
[
{
"type": "int",
"value": 19
}
],
[
{
"type": "int",
"value": 21
}
]
],
"result_hash": "sha256:c7fede3477f21b2f77b749c41badaa7465b6edd31397c900e37f5f6da611de83"
}
},
"plan_variant": {
"run": false,
"reason": "the plan variant was not asked for"
}
}The evidence records
gold: evidence-gold.json
SELECT DISTINCT team_fifa_api_id FROM Team_Attributes WHERE buildUpPlaySpeed > 50 AND buildUpPlaySpeed < 60- statement read from
- data/questions/mini_dev_postgresql.json
- digest
- sha256:d2731292f20b8d8569cd956dd747ffe1df13cd625076263e38ae9ebcef50b1ab
- origin
- https://bird-bench.oss-cn-beijing.aliyuncs.com/minidev.zip (sha256 cc48ba16838204e4e214512030cb572eeb5f7bcdd999bae4b9b6ff12ec13b92f, downloaded 2026-09-07), member minidev/MINIDEV/mini_dev_postgresql.json, 2024-06-19
result_hash sha256:1f7a83843d78e06ee35755d20fc3ce45b6c89aaf7eac10e855f720532e328e6b recomputed from this JSON: match
record_hash sha256:64867de833fb54d2d1784a27acb20326d86df1d25d36eb5e74541dc5675346ef recomputed from this JSON: match
the result this record holds, 161 rows
from evidence-gold.json, 161 rows
| team_fifa_api_idint8 |
|---|
| 485 |
| 110744 |
| 477 |
| 71 |
| 1889 |
| 68 |
| 229 |
| 481 |
| 52 |
| 70 |
| 80 |
| 1971 |
| 112225 |
| 1861 |
| 378 |
| 874 |
| 450 |
| 10030 |
| 1799 |
| 900 |
| 110745 |
| 1750 |
| 462 |
| 237 |
| 69 |
| 1824 |
| 1905 |
| 236 |
| 680 |
| 59 |
| 1848 |
| 1862 |
| 1844 |
| 100632 |
| 110374 |
| 665 |
| 452 |
| 200 |
| 682 |
| 674 |
| 44 |
| 88 |
| 100879 |
| 100087 |
| 82 |
| 111083 |
| 286 |
| 240 |
| 480 |
| 100741 |
| 1908 |
| 111989 |
| 650 |
| 15 |
| 48 |
| 457 |
| 1906 |
| 100805 |
| 19 |
| 81 |
| 244 |
| 160 |
| 232 |
| 77 |
| 1915 |
| 111087 |
| 21 |
| 468 |
| 3 |
| 17 |
| 1738 |
| 28 |
| 242 |
| 1796 |
| 111271 |
| 1832 |
| 456 |
| 165 |
| 1896 |
| 110502 |
| 100409 |
| 4 |
| 34 |
| 181 |
| 681 |
| 1747 |
| 1943 |
| 10020 |
| 10 |
| 673 |
| 1819 |
| 111974 |
| 472 |
| 86 |
| 110636 |
| 1913 |
| 144 |
| 1914 |
| 1904 |
| 1790 |
| 219 |
| 175 |
| 36 |
| 1739 |
| 31 |
| 1746 |
| 110329 |
| 109 |
| 873 |
| 13 |
| 1871 |
| 110569 |
| 459 |
| 62 |
| 246 |
| 1901 |
| 243 |
| 166 |
| 112512 |
| 206 |
| 46 |
| 110832 |
| 32 |
| 7 |
| 15005 |
| 247 |
| 1909 |
| 260 |
| 38 |
| 110749 |
| 479 |
| 1917 |
| 78 |
| 573 |
| 614 |
| 453 |
| 217 |
| 25 |
| 234 |
| 434 |
| 1902 |
| 111239 |
| 1867 |
| 1952 |
| 182 |
| 675 |
| 1860 |
| 1 |
| 106 |
| 449 |
| 1715 |
| 1853 |
| 110 |
| 110724 |
| 110747 |
| 100804 |
| 23 |
| 435 |
| 1907 |
| 898 |
| 203 |
what ran, and where
- run
- audit-eacac37d-7cc4-4d44-9121-67cecbf29b75
- executed at
- 2026-09-08T05:21:57.951758+00:00
- data as of
- 2026-09-08T05:21:47.344133+00:00
- backend at checkout
- PostgreSQL 16.15 (Debian 16.15-1.pgdg13+2) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 14.2.0-19) 14.2.0, 64-bit | server=172.17.0.2/32:5432 | database=bird
- backend that answered
- PostgreSQL 16.15 (Debian 16.15-1.pgdg13+2) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 14.2.0-19) 14.2.0, 64-bit | server=172.17.0.2/32:5432 | database=bird
- database role
- auditor
- replay rule
- R-SET
- question set version
- sha256:d2731292f20b8d8569cd956dd747ffe1df13cd625076263e38ae9ebcef50b1ab
- validator
- audit:libpg_query-parse
- checks run
- parses_as_exactly_one_statement, the_one_statement_is_a_select, no_placeholder_without_a_bound_parameter
- statement timeout
- 120000 ms
- rows
- 161 rows
the session it ran under
- engine
- postgresql
- time_zone
- Etc/UTC
- date_style
- ISO, MDY
- interval_style
- postgres
- extra_float_digits
- 1
- database_collation
- en_US.utf8
- work_mem
- 4096
- hash_mem_multiplier
- 2
recorded beside them
- statement_timeout
- 0
- search_path
- "$user", public
- server_version
- 16.15 (Debian 16.15-1.pgdg13+2)
- server_version_num
- 160015
- transaction_read_only
- on
- max_parallel_workers_per_gather
- 2
- server_encoding
- UTF8
- datlocprovider
- c
- daticulocale
- datcollversion
- 2.41
the rendering and the data
- version
- attestql/audit/2
- numeric_scale
- 6
- timestamp_format
- %Y-%m-%dT%H:%M:%S.%fZ
- timezone
- UTC
- null_rendering
- NULL
- encoding
- utf-8
- schema digest
- sha256:98bc0cba85d167faa0d5bb8e4db8801339559c93568b09cfd7c9a911f2c12123
- source file sha256
- sha256:31b1da211849d24a57c9af7636da46a5b82fc8a3ca1542bb3ebd8775e9a31cec
- rows in public.team_attributes
- 1458
second: evidence-second.json
SELECT team_fifa_api_id
FROM Team_Attributes
WHERE buildUpPlaySpeed > 50 AND buildUpPlaySpeed < 60- statement read from
- data/preds-pg/predict_mini_dev_gpt-4-32k_postgresql.json
- digest
- sha256:0fba1bfbeb487d641c8ae1e8bb01f762b2d29ddfb7c14dbd94e832f53cc10270
- origin
- https://raw.githubusercontent.com/bird-bench/mini_dev/b3d4bcbbae9a96934ad812551eb400c7a3b23c12/llm/exp_result/sql_output_kg/predict_mini_dev_gpt-4-32k_postgresql.json, 2024-06-19
result_hash sha256:1d5fb5460cba8d3b98439e43146df64c792a0e1c0f82b8cf5e922b2b48ba88c2 recomputed from this JSON: match
record_hash sha256:3b107386040ad8ac098ebbf7535cad0b028d5651b16ff96d1147385d3ed95eb5 recomputed from this JSON: match
the result this record holds, 356 rows
from evidence-second.json, 356 rows
| team_fifa_api_idint8 |
|---|
| 434 |
| 77 |
| 77 |
| 77 |
| 614 |
| 614 |
| 614 |
| 614 |
| 1901 |
| 1901 |
| 650 |
| 650 |
| 650 |
| 1861 |
| 229 |
| 229 |
| 229 |
| 229 |
| 111989 |
| 1 |
| 1 |
| 240 |
| 240 |
| 100409 |
| 100409 |
| 1906 |
| 1906 |
| 1848 |
| 1848 |
| 1848 |
| 32 |
| 21 |
| 675 |
| 1889 |
| 1889 |
| 234 |
| 88 |
| 88 |
| 88 |
| 88 |
| 3 |
| 3 |
| 3 |
| 160 |
| 4 |
| 4 |
| 4 |
| 4 |
| 4 |
| 59 |
| 23 |
| 1943 |
| 1896 |
| 1896 |
| 1896 |
| 378 |
| 378 |
| 378 |
| 1796 |
| 1796 |
| 450 |
| 450 |
| 450 |
| 450 |
| 78 |
| 78 |
| 1750 |
| 1750 |
| 1750 |
| 1867 |
| 110747 |
| 1799 |
| 1799 |
| 110502 |
| 242 |
| 242 |
| 242 |
| 242 |
| 242 |
| 110569 |
| 110569 |
| 110569 |
| 110569 |
| 110569 |
| 181 |
| 182 |
| 1824 |
| 1824 |
| 468 |
| 468 |
| 468 |
| 468 |
| 1746 |
| 452 |
| 10020 |
| 7 |
| 7 |
| 111271 |
| 111271 |
| 111271 |
| 1971 |
| 31 |
| 31 |
| 31 |
| 31 |
| 31 |
| 1915 |
| 1915 |
| 236 |
| 236 |
| 236 |
| 286 |
| 246 |
| 110374 |
| 110636 |
| 25 |
| 25 |
| 25 |
| 144 |
| 144 |
| 144 |
| 673 |
| 673 |
| 673 |
| 673 |
| 674 |
| 1860 |
| 110744 |
| 110744 |
| 110744 |
| 100632 |
| 100632 |
| 110832 |
| 110832 |
| 110832 |
| 110832 |
| 165 |
| 165 |
| 165 |
| 165 |
| 62 |
| 62 |
| 28 |
| 485 |
| 485 |
| 80 |
| 80 |
| 80 |
| 100879 |
| 100879 |
| 100879 |
| 100879 |
| 166 |
| 81 |
| 81 |
| 1952 |
| 1952 |
| 111239 |
| 111239 |
| 111239 |
| 111239 |
| 44 |
| 44 |
| 44 |
| 110745 |
| 110745 |
| 1832 |
| 1832 |
| 1832 |
| 1832 |
| 82 |
| 82 |
| 82 |
| 111083 |
| 110724 |
| 110724 |
| 110724 |
| 110724 |
| 472 |
| 1862 |
| 46 |
| 1738 |
| 1739 |
| 1739 |
| 873 |
| 1871 |
| 1871 |
| 1853 |
| 1853 |
| 1853 |
| 1844 |
| 1844 |
| 1844 |
| 217 |
| 573 |
| 453 |
| 453 |
| 10 |
| 10 |
| 219 |
| 219 |
| 219 |
| 68 |
| 68 |
| 69 |
| 69 |
| 69 |
| 69 |
| 1747 |
| 70 |
| 70 |
| 70 |
| 70 |
| 1904 |
| 71 |
| 13 |
| 13 |
| 13 |
| 13 |
| 13 |
| 112225 |
| 112225 |
| 477 |
| 682 |
| 682 |
| 479 |
| 100087 |
| 100087 |
| 100087 |
| 10030 |
| 1914 |
| 100741 |
| 200 |
| 111087 |
| 111087 |
| 1790 |
| 1790 |
| 247 |
| 15 |
| 15 |
| 15 |
| 15 |
| 456 |
| 86 |
| 86 |
| 480 |
| 449 |
| 449 |
| 449 |
| 449 |
| 243 |
| 243 |
| 457 |
| 203 |
| 203 |
| 203 |
| 203 |
| 1905 |
| 1902 |
| 52 |
| 874 |
| 874 |
| 874 |
| 1819 |
| 1819 |
| 111974 |
| 111974 |
| 111974 |
| 1913 |
| 1913 |
| 34 |
| 481 |
| 17 |
| 17 |
| 237 |
| 237 |
| 237 |
| 237 |
| 459 |
| 459 |
| 459 |
| 459 |
| 48 |
| 898 |
| 898 |
| 898 |
| 100805 |
| 110329 |
| 110329 |
| 110329 |
| 110329 |
| 100804 |
| 100804 |
| 680 |
| 680 |
| 232 |
| 232 |
| 232 |
| 232 |
| 106 |
| 106 |
| 260 |
| 260 |
| 260 |
| 1715 |
| 1715 |
| 1715 |
| 1715 |
| 1715 |
| 1908 |
| 462 |
| 462 |
| 462 |
| 462 |
| 462 |
| 206 |
| 206 |
| 36 |
| 36 |
| 36 |
| 1909 |
| 1909 |
| 665 |
| 665 |
| 665 |
| 665 |
| 38 |
| 38 |
| 109 |
| 109 |
| 19 |
| 681 |
| 681 |
| 1917 |
| 1917 |
| 1907 |
| 175 |
| 175 |
| 175 |
| 110 |
| 435 |
| 900 |
| 900 |
| 900 |
| 900 |
| 900 |
| 110749 |
| 110749 |
| 244 |
| 112512 |
| 112512 |
| 15005 |
| 15005 |
| 15005 |
| 15005 |
| 15005 |
what ran, and where
- run
- audit-eacac37d-7cc4-4d44-9121-67cecbf29b75
- executed at
- 2026-09-08T05:21:57.954527+00:00
- data as of
- 2026-09-08T05:21:47.344133+00:00
- backend at checkout
- PostgreSQL 16.15 (Debian 16.15-1.pgdg13+2) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 14.2.0-19) 14.2.0, 64-bit | server=172.17.0.2/32:5432 | database=bird
- backend that answered
- PostgreSQL 16.15 (Debian 16.15-1.pgdg13+2) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 14.2.0-19) 14.2.0, 64-bit | server=172.17.0.2/32:5432 | database=bird
- database role
- auditor
- replay rule
- R-SET
- question set version
- sha256:d2731292f20b8d8569cd956dd747ffe1df13cd625076263e38ae9ebcef50b1ab
- validator
- audit:libpg_query-parse
- checks run
- parses_as_exactly_one_statement, the_one_statement_is_a_select, no_placeholder_without_a_bound_parameter
- statement timeout
- 120000 ms
- rows
- 356 rows
the session it ran under
- engine
- postgresql
- time_zone
- Etc/UTC
- date_style
- ISO, MDY
- interval_style
- postgres
- extra_float_digits
- 1
- database_collation
- en_US.utf8
- work_mem
- 4096
- hash_mem_multiplier
- 2
recorded beside them
- statement_timeout
- 0
- search_path
- "$user", public
- server_version
- 16.15 (Debian 16.15-1.pgdg13+2)
- server_version_num
- 160015
- transaction_read_only
- on
- max_parallel_workers_per_gather
- 2
- server_encoding
- UTF8
- datlocprovider
- c
- daticulocale
- datcollversion
- 2.41
the rendering and the data
- version
- attestql/audit/2
- numeric_scale
- 6
- timestamp_format
- %Y-%m-%dT%H:%M:%S.%fZ
- timezone
- UTC
- null_rendering
- NULL
- encoding
- utf-8
- schema digest
- sha256:98bc0cba85d167faa0d5bb8e4db8801339559c93568b09cfd7c9a911f2c12123
- source file sha256
- sha256:31b1da211849d24a57c9af7636da46a5b82fc8a3ca1542bb3ebd8775e9a31cec
- rows in public.team_attributes
- 1458
Running these again
gold
re-run this statement read-only against PostgreSQL 16.15 (Debian 16.15-1.pgdg13+2) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 14.2.0-19) 14.2.0, 64-bit | server=172.17.0.2/32:5432 | database=bird under the session settings and over the data this record's fixture digest names, and compare the two results under R-SET
second
re-run this statement read-only against PostgreSQL 16.15 (Debian 16.15-1.pgdg13+2) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 14.2.0-19) 14.2.0, 64-bit | server=172.17.0.2/32:5432 | database=bird under the session settings and over the data this record's fixture digest names, and compare the two results under R-SET
This question's run
- run
- audit-eacac37d-7cc4-4d44-9121-67cecbf29b75
- server
- PostgreSQL 16.15 (Debian 16.15-1.pgdg13+2) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 14.2.0-19) 14.2.0, 64-bit | server=172.17.0.2/32:5432 | database=bird
- question set
- mini_dev_postgresql from https://bird-bench.oss-cn-beijing.aliyuncs.com/minidev.zip (sha256 cc48ba16838204e4e214512030cb572eeb5f7bcdd999bae4b9b6ff12ec13b92f, downloaded 2026-09-07), member minidev/MINIDEV/mini_dev_postgresql.json
- replay rule
- R-SET
The JSON this page was rendered from
- counterexample.json attestql/audit/counterexample/2
- evidence-gold.json the gold's evidence record
- evidence-second.json the prediction's evidence record
- smells.json attestql/audit/smells/2