Suite
Tasks
A task is a manifest that names a fixture and a request, plus a list of assertions drawn from the vocabulary insection 6 of the standard. The runner has no task-specific code; everything it needs is in those two files. Before a task is used it is run twice without a model, once making no changes (which has to fail) and once with the reference solution (which has to pass). The latest result of each run is in the last two columns.
| Task | Tier | Mode | Expected | Fixture | Assertions | No-op run | Reference solution |
|---|---|---|---|---|---|---|---|
| do-budget-refusal-pullup | medium | do | refusal | antmicro-microphone-board | 9 (9 req.) | fail | pass |
| do-rename-net | simple | do | edit | antmicro-microphone-board | 13 (13 req.) | fail | pass |
Each task page has the README, the manifest, the full assertion list with a note on what each check is for, and every result record for that task.Adding a task