Task · simple tier · do mode · expects edit

do-rename-net

Tier: simple · Mode: do · Expected outcome: edit · Covers: AC-3.1, AC-3.7 Fixture: antmicro-microphone-board (Antmicro PDM microphone board, Apache-2.0)

What this tests

A propagating rename on a real board. DATA is one of three signal nets on the microphone breakout, alongside CLK and SELECT. It appears on two labels in the 8,489-line schematic, and on the scaffolded docs/PINOUT.md. The correct run renames it everywhere, updates the docs, introduces no new verification violation, and commits once.

The trap

Three, pulling in different directions.

Partial propagation. A model that edits the first occurrence and stops leaves a schematic that still loads and may raise no new error, because an orphaned label is not necessarily one. old-net-gone catches it, which is why the assertion is stated as absence of the old name rather than presence of the new one.

Regeneration. The cheapest way to guarantee a consistent rename is to rewrite the file. That satisfies every connectivity and document assertion here, and it is exactly what copperhead exists to prevent: a regenerated .kicad_sch loses hand placement, UUIDs, and any hope of a reviewable diff. surgical-schematic-edit is the only assertion that fails it.

Over-reach. DATA is a common substring, and the board has neighbouring signal nets. A model doing textual replacement without understanding scope can catch more than it was asked to. sibling-nets-untouched is the sentinel.

Why the diff bound is 1 percent, not 5

AC-3.7 states surgicality as 5 percent of a file’s lines. That number was calibrated against a small synthetic schematic where 5 percent is about fifteen lines and therefore binding.

This schematic is 8,489 lines. Five percent is 424 lines: enough to rewrite whole subsystems while passing. A correct rename here touches roughly six lines, so the bound is set at 1 percent, allowing 84. That is generous for a careful run that also tidies a comment, and unreachable for one that regenerates.

The general rule is in FIXTURES.md section 6: a surgicality bound is calibrated per task against the file it constrains, not inherited as a constant.

Why verification is asserted baseline-relative

This fixture carries 44 ERC warnings and 13 DRC warnings at baseline, all of them library resolution: the design references Antmicro’s own symbol and footprint libraries, which are not vendored here. Zero errors, zero unconnected items, zero parity issues.

A strict erc_clean assertion would fail every run on this fixture regardless of what the model did. erc_no_new_violations compares against the recorded baseline and fails only on something the run introduced, which is the question the task is actually asking.

Reproducing by hand

FIX=$(mktemp -d) && cp -r fixtures/antmicro-microphone-board/tree/* "$FIX/"
git -C "$FIX" init -q && npm run dev -- --repo "$FIX" init
git -C "$FIX" add -A && git -C "$FIX" commit -qm baseline
npm run dev -- --repo "$FIX" do "rename net DATA to PDM_DATA"

The runner does the same in a temporary sandbox, three times per model, with the response cache off.

This page is rendered from tasks/do-rename-net/README.md in the repository, at the commit the site was built from.

Manifest

The contents of task.json. Every result record stores a hash of this file, so records made against an earlier version of the task are kept separate from records made against this one.

Request
rename net DATA to PDM_DATA
Fixtureantmicro-microphone-board dad443ad059b7cd2ec57313967db7c511caa8bcfd1ba72dc2f02cd3560249529
Tiersimple
Modedo
Expected outcomeedit
Setupcopperhead init
Capsturns: 40 wallClockSec: 1200
Config
{
  "maxTurns": 40,
  "maxRepairCycles": 5
}
TagsAC-3.1, AC-3.7, propagation, surgicality, docs-memory
Manifest hash524b94d28beb7e1c28af0659c77b59a236687980a8b06f89aaab3d7a5ff151bb
Sourcetask.json · assertions.json

Assertions

13 assertions, 13 of them required. A run passes when all of the required ones pass. Weights are only used for partial credit, which isn't shown on the leaderboard. The last column explains what each check is looking for.

IdTypeArgumentsWeightRequiredWhat it catches
new-net-existsnet_presentnet: "PDM_DATA"3yesThe requested change actually happened. Without this, a run that verifies and commits while doing nothing scores as a pass.
old-net-gonenet_absentnet: "DATA"3yesA rename that leaves the old label behind is a copy. DATA appears on two labels in this schematic, so partial propagation is the likely failure.
sibling-nets-untouchednet_presentnet: "CLK"1yesCLK and SELECT are the neighbouring signals. A rename that catches them too is over-reach, and CLK is the cheapest sentinel for it.
no-new-erc-violationserc_no_new_violationsseverity: "error"2yesBaseline-relative: this fixture carries 44 library-resolution warnings it cannot resolve in a bare checkout. Strict erc_clean would fail every run for reasons unrelated to the model.
no-new-drc-violationsdrc_no_new_violationsseverity: "error"1yesBaseline unconnected items and schematic parity are both zero, so a net rename that desyncs the board shows up here.
pinout-updateddoc_containsdoc: "PINOUT.md" pattern: "PDM_DATA"1yes
pinout-old-name-gonedoc_containsdoc: "PINOUT.md" pattern: "\\bDATA\\b" absent: true1yes
surgical-schematic-editdiff_ratio_maxpath: "hardware/microphone-board.kicad_sch" ratio: 0.013yesAC-3.7, calibrated to this file. A correct rename touches roughly six lines of 8,489; 1 percent allows 84, which is generous for a careful run and impossible for a regenerated one. The AC's nominal 5 percent would allow 424 lines here.
board-untouchedfile_unchangedpath: "hardware/microphone-board.kicad_pcb"1yesNet names live in the schematic. Rewriting a 35,688-line board file for a label change is going looking for work.
touched-files-boundedfiles_touched_subsetallowed: ["hardware/microphone-board.kicad_sch","docs/*.md",".copperhead/**"]2yes
one-commitcommit_countequals: 11yes
finished-cleanlyexit_path_inpaths: ["done"]2yes
no-secretno_secret1yes

Discrimination check

The most recent no-op and reference-solution runs for this task. The no-op should fail and the reference solution should pass. If either didn't, the cell shows which assertion was responsible.

TaskTierExpectedNo-op runReference solutionDate
do-rename-netsimpleeditfail new-net-existspass2026-09-07

Records

8 records for this task. Rows in noop or gold mode are the suite's own checks and don't involve a model.

RecordDateModelModeRepeatVerdictFirst failed requiredComparable
2026-09-07/harness-gold/do-rename-net/run-1.json2026-09-07harness:goldgold1passyes
2026-09-07/harness-noop/do-rename-net/run-1.json2026-09-07harness:noopnoop1failnew-net-existsyes
2026-09-06/harness-gold/do-rename-net/run-1.json2026-09-06harness:goldgold1unscoreableno-new-erc-violationsno: suiteVersion 0.1.0 ≠ 0.2.0; setup skipped: init
2026-09-06/harness-gold/do-rename-net/run-2.json2026-09-06harness:goldgold2passno: suiteVersion 0.1.0 ≠ 0.2.0; setup skipped: init
2026-09-06/harness-gold/do-rename-net/run-3.json2026-09-06harness:goldgold3passno: suiteVersion 0.1.0 ≠ 0.2.0
2026-09-06/harness-noop/do-rename-net/run-1.json2026-09-06harness:noopnoop1failnew-net-existsno: suiteVersion 0.1.0 ≠ 0.2.0; setup skipped: init
2026-09-06/harness-noop/do-rename-net/run-2.json2026-09-06harness:noopnoop2failnew-net-existsno: suiteVersion 0.1.0 ≠ 0.2.0; setup skipped: init
2026-09-06/harness-noop/do-rename-net/run-3.json2026-09-06harness:noopnoop3failnew-net-existsno: suiteVersion 0.1.0 ≠ 0.2.0