Development record
a-person-sets-how-many-models-review
Summary
The build loop's validate stage stops running one fixed reviewer per code-review role and runs a review set instead: a list of reviewers per role, each at a model class, and an optional judge. The bundled default is two reviewers per role, the middle class and the class below it, with the stricter verdict deciding (the product thinker, 2026-10-24). Every round records, per reviewer, its class, the model asked for and the model reported, its verdict and its located findings, with whether the fix round applied or rejected each, so a later choice of review set is made from counts.
Approach, by acceptance criterion:
- Configuration.
review.reviewers(a list of classes, one or two entries, per role or for both) andreview.judge(a class or absent) resolve through the layers the pace uses (flag, repository, machine, bundled), read once when the run starts and kept for the run. The repository's list is clamped toreview.max_reviewersfrom the machine's config (two when unset), with one stderr line naming both files. The class mapmodels.classes(top,middle,below,smallest, each an admitted<provider>/<model>route) is read from the machine's config or the bundled default alone; a repository value there is skipped with a warning, as a repository runner route is. A class the map lacks, or a judge whose agent has no passing calibration record under itd-81, is refused at the existingreviewstage (internal/core/implement/loop/validate.go,StageReview) before anything is written. The run'spacepayload gains areviewsibling naming each value's layer. - Per-request model.
runner.Requestgains a model, admitted against the provider allowlist as a route model is, so one runner serves two reviewers at two classes; the claude and opencode runners pass it on their command line. A host-run reviewer is handed the class and the resolved model in itsawaitingpayload; the record marks its model as asked, not confirmed. - Rounds keyed by reviewer. A round's unit becomes (role, reviewer index): round
directories, briefs, returns and the receipt match gain the index, so two reviewers of
one role never collide; the briefs differ only in their return path. The run state's
schema version is bumped, and an older state is migrated (its single reviewer becomes
index 0) or refused by the loop's existing version rules. Reviewers are handed out in
consecutive steps within
sub_agents, each taking a slot, on the lane's head. - Verdict. Disagreement is one reviewer's verdict passing and the other's failing, by the existing pass bit of each verdict word. The round fails when any reviewer fails it, as today: that is the stricter verdict.
- Findings record. Each
file:linecited under a return's findings is one located finding of that reviewer; a return with none records zero located findings and is not refused. After the fix round, each finding carries applied or rejected from the fix report, orunknownwhen the report does not say. Two findings at the same file and line are marked as cited by both.implement record --jsonandimplement statusrender them. - Judge. On a disagreement with a judge set, the judge is handed both returns and the head and may only strike findings it rules false from the fix brief, each with its reason; it never passes a round, and a round whose failing findings are all struck goes to a fix round with an empty brief. Until a judge agent clears itd-81's gate the judge cannot be enabled, so the default runs without one.
Out of scope, recorded: the other top-class consult cases and the writer-class ruling (iss-2610240946046901); reviews started by hand outside the loop (itd-83's seam); the intent-auditor's fan-out.
Footprint
- packages: internal/core/implement/loop, internal/core/runner, internal/core/layered, internal/surface/cli, commands
- tests: internal/core/implement/loop review-set resolution, clamp and refusal tests; per-reviewer round path and state-migration tests; disagreement and stricter-verdict tests; findings extraction, applied or rejected and overlap tests; judge strike-only tests; internal/core/runner per-request model admission tests
Steps
- The review set and the class map
- packages: internal/core/implement/loop, internal/core/layered, internal/surface/cli, commands
- tests: the bundled default resolves to two reviewers per role (middle, below); a repository list above the machine's ceiling is clamped with one stderr line naming both files; a repository class map is skipped with a warning; an unknown class and an uncalibrated judge are each refused at the
reviewstage with nothing written; a running run keeps the set it started with; the payload names each value's layer - needs: none
- landed: a7ab8927bf0b
- A runner takes a model per request
- packages: internal/core/runner
- tests: a request's model is admitted against the provider allowlist and refused off it before the runner starts; the claude and opencode runners pass the request's model and record it as asked beside the model reported; a request without one keeps the route's model
- needs: none
- landed: 9163555347f5
- Rounds keyed by reviewer, and the stricter verdict across them
- packages: internal/core/implement/loop, commands
- tests: two reviewers of one role get distinct round paths, briefs differing only in the return path, and receipts matched by index; an older run state migrates its reviewer to index 0 or is refused by version; both reviewers are handed out within the sub-agent ceiling; a host-run reviewer's model is recorded as asked, not confirmed; pass against fail is a disagreement, two failing verdicts are not, and any failing reviewer fails the round
- needs: 1, 2
- rulings: dcn-2610292236161673 (a request for a model the machine pins is refused on claude with no fallback; a request model and a --route model that differ are refused before launch, naming both)
- landed: b22578aee474
- The per-reviewer findings record
- packages: internal/core/implement/loop, internal/surface/cli, commands
- tests: each
file:lineunder a return's findings is one located finding of its reviewer; a return with none records zero and is not refused; the fix report's applied or rejected status is joined to each finding,unknownwhere it is silent; two findings at the same file and line are marked as cited by both;implement record --jsoncarries them - needs: 3
- The judge strikes false findings only
- packages: internal/core/implement/loop, commands
- tests: with a calibrated judge fixture, a disagreement hands the judge both returns and the head; a struck finding leaves the fix brief with its reason recorded; the judge never passes a round; a round whose failing findings are all struck still opens a fix round with an empty brief; agreeing reviewers start no judge
- needs: 3
Frontmatter
| id | spc-2610240951109420 |
|---|---|
| type | spec |
| lifecycle | open |
.abcd/development/specs/open/spc-2610240951109420-a-person-sets-how-many-models-review.md
open on GitHub ↗ · commit history ↗
References1
- itd-2610240922461281The review set is a setting, in model classesimplements
mentions8
- dcn-26102412565473452026-10-24 — The build of itd-2610240922461281 (spc-2610240951109420, the review set as a setting) waits until two things hold: this records branch has merged, because the build's brief stage refuses an intent that main does not carry as planned; and the drain's live reservation on internal/core/implement/loop (iss-2610142134353345) is released, because the spec's footprint names that package and two builds whose footprints overlap back off. It is deferred, not dropped: the session that planned it starts it once both hold, with one sub-agent at a time and one preflight at a time, as the 2026-10-24 machine-load rules require.
- dcn-26102922361616732026-10-29 — Decided by the technical facilitator on delegation (coordinator rulings on the review of the merge that landed step 2 of spc-2610240951109420, 2026-10-29), two rules for a model asked per request, for step 3 to build. (a) When a request asks the claude runner for a model the machine pins, the dispatch is refused before anything launches and nothing falls back, to the host or to any other runner, with the words the refusal of a --route to a pinned model on claude already uses: claude can hold no pin, and a run on another model is a run nobody asked for. As merged, the request is refused at launch and the role is handed back to the host; step 3 makes that refusal final. (b) When a request names a model and a --route names a model too, and the two differ, the dispatch is refused before anything launches, naming both; when they are the same model it runs, and the up-front pin check vets that one model.
- iss-2610240946046901Four of the product thinker's rulings of 2026-10-14 in iss-2610140652230444 are carried by no record once itd-2610240922461281 takes the review set: the top class consulted (1) for a third read of release-blocking security changes (the shell guard, installing and updating the binary, secrets, what is published), (2) before a design-heavy plan locks, and (3) when the executor fails twice with the same failure fingerprint; and (4) code is written by the middle class. The review-set intent carries only the fourth consult case (the pair disagrees), as its judge. Each of the four sits at a seam outside the review round (a release gate, the planning interview, the fix-round loop, the implementer's route), so each is its own piece of work; deferred out loud here rather than left inside a draft's Decisions where a reader of the issue would not find it.
- iss-2611020719494343A lane whose change raises the run state's schema rewrites the live run's state at the new version: its own preflight's load gate (implement load --wait, run through the lane's freshly built binary) renews the run's reservation and saves the run state with the lane binary's schema, after which the driver, still on the default branch's older abcd, refuses every step and receipt ('schema version 18, which a newer abcd wrote'). On 2026-11-02 the lane for iss-2611012047534449, which moves the schema from 17 to 18, did this three times to its own run (about 04:56, 05:57 and 07:15 UTC, the last in its landing preflight), and the version was set back by hand each time, no version-18 field being in use.
- iss-2611030744332925Older build runs landed spec steps without marking them landed: on 2026-11-03 four open specs held merged steps with no landed line (spc-2610240951109420 steps 1-2 from #1145 and #1193, spc-2610161949559050 steps 1-2 from #1179 and #1187, spc-2609301811532881 steps 1-6 from #1110 and #1197, spc-2610230730269295 step 1 from #1197), so the board offered them as ready from step 1 and abcd build would have rebuilt delivered work. Found only by grepping merge messages.
- iss-2611031719593367Two notes from #1378's landing review (spc-2610240951109420 step 3, 2026-11-03): handedMove in internal/core/implement/loop/loop.go tells the host two different things about which model to start for a host-run review-set reviewer under tiering; and holdRequestModel in internal/core/runner/config.go vets only the route's own runner, so a launch-time fallback to another runner is not re-held against the requested model (unconfirmed).
- itd-81No Judge Ships Unmeasured, And A Clean Diff Is A Result
- itd-83The Review Bar Fires By Itself, In Every Repo abcd Manages