Evidence
Result records
One record per task, model and repeat. Records are written once and not edited afterwards. Each one holds the verdict, the outcome and detail of every assertion, and the stamps used to decide comparability: schema and suite version, task manifest hash, fixture hash, copperhead and kicad-cli versions, Node version and platform. The leaderboard is computed from these files alone, and anyone can re-score them offline.
Snapshot
16 records0 agent records, 16 harness records.4 comparable with the suite as checked out, 12 not.Dates 2026-09-06 to 2026-09-07.
Aggregated leaderboard as JSONComparable means
suite 0.2.0Schema 1.0.0, suite 0.2.0, the checked-out task manifest and fixture hashes, andkicad-cli 10. A record that doesn't match on one of these, or whose run skipped a setup command, is listed with the reason and left out of the leaderboard.
Raw files on GitHub| Record | Date | Model | Task | Mode | Repeat | Verdict | First failed required | Comparable |
|---|---|---|---|---|---|---|---|---|
| 2026-09-07/harness-gold/do-budget-refusal-pullup/run-1.json | 2026-09-07 | harness:gold | do-budget-refusal-pullup | gold | 1 | pass | yes | |
| 2026-09-07/harness-gold/do-rename-net/run-1.json | 2026-09-07 | harness:gold | do-rename-net | gold | 1 | pass | yes | |
| 2026-09-07/harness-noop/do-budget-refusal-pullup/run-1.json | 2026-09-07 | harness:noop | do-budget-refusal-pullup | noop | 1 | fail | refused | yes |
| 2026-09-07/harness-noop/do-rename-net/run-1.json | 2026-09-07 | harness:noop | do-rename-net | noop | 1 | fail | new-net-exists | yes |
| 2026-09-06/harness-gold/do-budget-refusal-pullup/run-1.json | 2026-09-06 | harness:gold | do-budget-refusal-pullup | gold | 1 | pass | no: suiteVersion 0.1.0 ≠ 0.2.0; setup skipped: init | |
| 2026-09-06/harness-gold/do-budget-refusal-pullup/run-2.json | 2026-09-06 | harness:gold | do-budget-refusal-pullup | gold | 2 | pass | no: suiteVersion 0.1.0 ≠ 0.2.0; setup skipped: init | |
| 2026-09-06/harness-gold/do-budget-refusal-pullup/run-3.json | 2026-09-06 | harness:gold | do-budget-refusal-pullup | gold | 3 | pass | no: suiteVersion 0.1.0 ≠ 0.2.0 | |
| 2026-09-06/harness-gold/do-rename-net/run-1.json | 2026-09-06 | harness:gold | do-rename-net | gold | 1 | unscoreable | no-new-erc-violations | no: suiteVersion 0.1.0 ≠ 0.2.0; setup skipped: init |
| 2026-09-06/harness-gold/do-rename-net/run-2.json | 2026-09-06 | harness:gold | do-rename-net | gold | 2 | pass | no: suiteVersion 0.1.0 ≠ 0.2.0; setup skipped: init | |
| 2026-09-06/harness-gold/do-rename-net/run-3.json | 2026-09-06 | harness:gold | do-rename-net | gold | 3 | pass | no: suiteVersion 0.1.0 ≠ 0.2.0 | |
| 2026-09-06/harness-noop/do-budget-refusal-pullup/run-1.json | 2026-09-06 | harness:noop | do-budget-refusal-pullup | noop | 1 | fail | refused | no: suiteVersion 0.1.0 ≠ 0.2.0; setup skipped: init |
| 2026-09-06/harness-noop/do-budget-refusal-pullup/run-2.json | 2026-09-06 | harness:noop | do-budget-refusal-pullup | noop | 2 | fail | refused | no: suiteVersion 0.1.0 ≠ 0.2.0; setup skipped: init |
| 2026-09-06/harness-noop/do-budget-refusal-pullup/run-3.json | 2026-09-06 | harness:noop | do-budget-refusal-pullup | noop | 3 | fail | refused | no: suiteVersion 0.1.0 ≠ 0.2.0 |
| 2026-09-06/harness-noop/do-rename-net/run-1.json | 2026-09-06 | harness:noop | do-rename-net | noop | 1 | fail | new-net-exists | no: suiteVersion 0.1.0 ≠ 0.2.0; setup skipped: init |
| 2026-09-06/harness-noop/do-rename-net/run-2.json | 2026-09-06 | harness:noop | do-rename-net | noop | 2 | fail | new-net-exists | no: suiteVersion 0.1.0 ≠ 0.2.0; setup skipped: init |
| 2026-09-06/harness-noop/do-rename-net/run-3.json | 2026-09-06 | harness:noop | do-rename-net | noop | 3 | fail | new-net-exists | no: suiteVersion 0.1.0 ≠ 0.2.0 |