Development record
The pre-push gate is blind to both eval lanes, so the read-block eval is guarded by nothing that blocks anything. make preflight runs the six lint gates, go build, go vet, go test over the untagged packages and the race lane, and neither make smoke nor make evals-cold-reading; the eval files sit behind a build tag, so the untagged test run does not compile them. A defect in the cold-reading evals therefore passes every local gate and surfaces only in CI. That was not hypothetical: a path-elision defect in the amnesia eval's own guard was unsatisfiable wherever the process temp directory is the Linux one, so it would have landed green locally and red in the merge queue, and it was found by an adversarial review rather than by any gate. Pair this with the CI job not being a required status check and the position is that the eval which certifies the firewall runs in no gate that can stop anything: preflight does not execute it, and the job that does execute it cannot block a merge.
The pre-push gate is blind to both eval lanes, so the read-block eval is guarded by nothing that blocks anything. make preflight runs the six lint gates, go build, go vet, go test over the untagged packages and the race lane, and neither make smoke nor make evals-cold-reading; the eval files sit behind a build tag, so the untagged test run does not compile them. A defect in the cold-reading evals therefore passes every local gate and surfaces only in CI. That was not hypothetical: a path-elision defect in the amnesia eval's own guard was unsatisfiable wherever the process temp directory is the Linux one, so it would have landed green locally and red in the merge queue, and it was found by an adversarial review rather than by any gate. Pair this with the CI job not being a required status check and the position is that the eval which certifies the firewall runs in no gate that can stop anything: preflight does not execute it, and the job that does execute it cannot block a merge.
Grounds
- pursued: a defect in the read-block or amnesia eval now fails locally before it reaches CI; it would be shown wrong by a preflight run that is green while make evals-cold-reading or make smoke is red.
Correction, 2026-09-29
The test the resolution names, TestPreflightRunsBothEvalLanes, is
TestPreflightRunsEveryTaggedEvalLane in internal/core/lint/preflightgates_test.go
since 3eb4b549, which derives the eval lanes from the Makefile rather than
naming two. The rename changes nothing the resolution claims: the pre-push gate
still runs every tagged eval lane.
Frontmatter
| id | iss-2608311632382737 |
|---|---|
| type | issue |
| lifecycle | resolved |
| severity | major |
.abcd/work/issues/resolved/iss-2608311632382737-the-pre-push-gate-is-blind-to-both-eval.md
open on GitHub ↗ · commit history ↗
mentions5
- iss-2609021152026246make preflight runs record-lint without -agent-diff, so the agent_contract sub-check that asks whether a changed agent prompt bumped its prompt_version and wrote its agents/CHANGELOG.md entry is a no-op locally and arms only in CI, which passes -agent-diff. PR 606 passed a green local preflight three times and was refused in the merge queue for exactly that check: agents/cold-reading-widening.md changed with prompt_version still 0.1.0. Same shape as iss-2608311632382737 (preflight blind to the eval lanes, fixed by PR 607): a gate the push relies on that the local run cannot see. Fix: preflight passes -agent-diff origin/main...HEAD (or the merge-base) to record-lint, and the gate-roster test that pins preflight's contents asserts it.
- iss-2609261034583909Eight resolved records name, in their resolution notes, a test no longer in the tree; each was true when written, and RS006 reads only records entering a terminal folder, so none is refused, but a reader following the note finds nothing. Verified at b7646ff1, with the commit that removed each test: iss-184 names TestTokenizeRejectsUnterminatedHeredoc (16d50feb renamed it TestTokenizeFlagsUnterminatedHeredoc and changed the behaviour: an unterminated heredoc is a verdict, not ErrUnparsableCommand, so the note's part (2) is stale too); iss-2608301808193750 names TestIsAbsentValueIsASpellingTestNotANullTest (3eb4b549 replaced the spelling predicate with a class-based one, TestAbsenceIsDecidedByClassNotBySpelling, so the note's 'deliberately NOT widened' is superseded); iss-2608311632382737 names TestPreflightRunsBothEvalLanes (the test is TestPreflightRunsEveryTaggedEvalLane, which derives the lanes from the Makefile; the one clean rename of the eight); iss-2608311632439831 names TestComparativeRefusesToAssemble (3b62c967 made the comparative position assemble from the widening run's items, so the refusal the note describes no longer exists); iss-2609012039117381 names TestBinaryHooksFallBackToAPathBinary and iss-275 names TestGuardShimFallsBackToPATH (c637a734 removed the PATH rung those tests pinned: the shims run only the abcd the machine recorded); iss-354 names TestWorkflowGoVersionsMatchSubstitutions (removed in df377ee6, replaced by TestEverySetupGoResolvesTheToolchainFromGoMod under iss-2609090951291799); iss-2609090951291799 names TestWorkflowGoVersionsMatchSubstitutions only to say it was replaced, which stays accurate. Six of the eight need an amendment note stating what superseded the test and the claim, not a rename, so the set is not a mechanical fix.
- itd-2609020625400445A committed preset declares the window it was calibrated for, and the record holds every preset to its declaration
- spc-68The size report: a per-kind cost on every assembly, a test kind split from source, and a version that cannot misreport its table
- spc-2609020626048722One committed entry per position, in three parts: the object set, the kinds within it and the window it measures at, held by an eval that re-measures every entry on every committed change