Development record
Measured on 2026-09-01 with thirteen auto-merge pull requests in flight: each one pays two full CI cycles before it lands, and the macOS check job alone takes about 13 minutes (ubuntu 9). Cycle one: the pull request is armed, main moves, the strict up-to-date policy makes it BEHIND, the keep-current script updates the branch, and the full CI re-runs on the updated head before the pull request is CLEAN enough to enter the merge queue. Cycle two: the merge queue runs the full CI again on the merge group. Every merge moves main and knocks the not-yet-queued pull requests back to cycle one, so a batch of thirteen cost roughly twenty-six 13-minute cycles serialised in ALLGREEN groups, and a one-line record change waited an hour. The maintainer asks how to speed the gates up, for example by running them locally first. Directions, none adopted: (1) local-first is already built and not wired on every account: make preflight is the pre-push gate and .githooks/pre-push runs it, but core.hooksPath is unset on at least one active account, so nothing runs before a push; wire it at ahoy install (an owned ConfigChange) and record which accounts have it; note that a local pass shortens nothing on the forge, it only stops red pushes. (2) Run the full matrix once, in the queue: on the pull_request event run the fast lane only (format, record gates, ubuntu build and test) and keep the macOS leg, the race lane and the smoke harness for the merge_group event, which already runs everything; the CI classifier that stands macOS down for docs-only changes shows the seam exists. (3) Cache the Go build and test cache across runs (actions/setup-go cache keyed on go.sum) and check whether the macOS job's 13 minutes is test time or cold-build time. (4) Let the queue batch: min_entries_to_merge_wait_minutes is 0, so each pull request tends to get its own group; a short wait lets several share one CI run. (5) The strict policy is the multiplier and was kept on 2026-09-01 (iss-2609012202237613); revisit only with the duplicate-id gate argument answered.
Measured on 2026-09-01 with thirteen auto-merge pull requests in flight: each one pays two full CI cycles before it lands, and the macOS check job alone takes about 13 minutes (ubuntu 9). Cycle one: the pull request is armed, main moves, the strict up-to-date policy makes it BEHIND, the keep-current script updates the branch, and the full CI re-runs on the updated head before the pull request is CLEAN enough to enter the merge queue. Cycle two: the merge queue runs the full CI again on the merge group. Every merge moves main and knocks the not-yet-queued pull requests back to cycle one, so a batch of thirteen cost roughly twenty-six 13-minute cycles serialised in ALLGREEN groups, and a one-line record change waited an hour. The maintainer asks how to speed the gates up, for example by running them locally first. Directions, none adopted: (1) local-first is already built and not wired on every account: make preflight is the pre-push gate and .githooks/pre-push runs it, but core.hooksPath is unset on at least one active account, so nothing runs before a push; wire it at ahoy install (an owned ConfigChange) and record which accounts have it; note that a local pass shortens nothing on the forge, it only stops red pushes. (2) Run the full matrix once, in the queue: on the pull_request event run the fast lane only (format, record gates, ubuntu build and test) and keep the macOS leg, the race lane and the smoke harness for the merge_group event, which already runs everything; the CI classifier that stands macOS down for docs-only changes shows the seam exists. (3) Cache the Go build and test cache across runs (actions/setup-go cache keyed on go.sum) and check whether the macOS job's 13 minutes is test time or cold-build time. (4) Let the queue batch: min_entries_to_merge_wait_minutes is 0, so each pull request tends to get its own group; a short wait lets several share one CI run. (5) The strict policy is the multiplier and was kept on 2026-09-01 (iss-2609012202237613); revisit only with the duplicate-id gate argument answered.
Deferral 2026-09-29
Deferred past v0.11.1: The product thinker's ruling M19 of 2026-09-23: planned next cycle as its own intent that chooses among the record's directions, not folded into itd-115. Since filing, the pre-push hook checks a preflight receipt instead of running the preflight (2026-09-25) and the macOS leg's cap rose to 45 minutes (ruling Z, 2026-09-28); neither removes the second CI cycle. Owed: that intent, which opens on one question: which direction first, hooksPath at install, the full matrix in the queue only, caching, or queue batching?
Remedy grounds (2026-09-29)
- Each direction is stated so the interview can pick one without further research; the measure is the record's own (two full cycles per pull request).
- SOTA check: GitHub's merge-queue documentation (https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue, read 2026-09-29) gives the minimum group size and its wait time for lengthy CI, and says the queue provides the benefits of the up-to-date requirement without the author updating the branch, which bears on direction 5 once the duplicate-id argument is answered; actions/setup-go (https://github.com/actions/setup-go, read 2026-09-29) enables caching by default, so direction 3 is largely in place.
- Rejected: dropping the strict policy now, which the record defers to the duplicate-id argument.
Evidence 2026-10-07
The macOS leg hit its 45-minute job cap in the merge queue again. On #839's merge-group run (37590582904), check (macos-latest) ran 07:57:44 to 08:43:14 and was cancelled in Test (race, internal), which had started at 08:14:47. The ubuntu leg was green. The queue dropped the pull request and it had to be re-queued by hand, adding a full cycle. This is the second time in two release cycles (the v0.13.1 release PR, 2026-10-05, was cancelled the same way). The queue retries nothing on a runner timeout, so an unattended run has to watch for it.
Evidence 2026-10-08
The macOS leg's 45-minute cap recurred three more times in the overnight drain, though iss-2609281514435020 was resolved for the same class. PR #852's own macOS check failed at 45m31s (2026-10-07 23:11Z). PR #854 was dropped from the merge queue at 2026-10-08 00:36Z on a failed merge-group check, its own checks green. PR #867 was dropped at 07:43Z: its merge-group ci run 37740516314 had check (macos-latest) cancelled at the cap (07:07 to 07:52). Each needed a hand re-run or re-enqueue (iss-2610090642376032 covers the silent drop).
Measured again (2026-10-11, CI-cycles research). Over the last 300 runs of ci (2026-10-05 to 2026-10-11, 71 merged pull requests), a merged pull request took 1.07 pull_request runs on average and only 6 of 71 took a second, so the strict policy's BEHIND re-run is now rare; the surplus is in the queue instead (131 merge_group runs, 1.27 per merged pull request by its own number, 15 of 71 queued more than once) and in the push to main (68 runs). GitHub documents that the merge queue "provides the same benefits as the Require branches to be up to date before merging" setting, and "Merge limits do not combine merge_group builds", so direction 4's longer wait cuts no runs. Turning strict off still needs the product thinker's ruling, which kept it on 2026-09-01; at these numbers it would save about 6 runs a week. The macOS leg, the scarce runner, is taken out of the pull-request run by iss-2610110938282654 either way.
Frontmatter
| id | iss-2609012313465609 |
|---|---|
| type | issue |
| lifecycle | open |
| severity | major |
.abcd/work/issues/open/iss-2609012313465609-every-pull-request-pays-two-full-ci.md
open on GitHub ↗ · commit history ↗
References1
- itd-2610261025516272Private repos land checked work without renting CIrelated
mentions6
- iss-2609012202237613Every auto-merge pull request opened tonight (#571, #572, #574) sat BEHIND with queue entry null until a hand-run update-branch, while #570 and #573 entered the queue only because they happened to be up to date at the moment they were armed. Cause: main-protection combines the merge queue (grouping ALLGREEN, merge method MERGE, mirrored 2026-08-19) with strict_required_status_checks_policy true. Auto-merge enqueues a pull request only when its merge state is CLEAN, and under the strict policy a pull request behind main is never CLEAN, so it cannot enter the queue whose whole purpose is to test it combined with the current base. Every merge from the queue moves main and knocks the next un-queued pull request BEHIND again, so a batch of parallel PRs needs one manual update per merge. iss-172 records the invariant that strict is never relaxed because it is what gates the merged result against a duplicate record id minted by two checkouts; that reasoning predates the queue, and the queue now gates exactly that merged result (the group commit on top of main runs every required check before anything lands), so strict adds no gate the queue does not already hold and costs the queue its entry. itd-115 promises the queue removes this toil. Directions, for the maintainer to rule: (1) set strict false in the live ruleset and the mirror, keeping the queue as the gate, and record the ruling against iss-172's invariant; (2) keep strict and automate iss-172 rung 1, an update-branch loop after arming auto-merge, which is what tonight's session runs by hand.
- iss-2609281514435020The merge queue fails a group whose macOS check job runs past 30 minutes, cancelling pull requests whose code is fine: #728 was cancelled at 30.3 minutes, #730 twice at the cap, and #733 once. The check job's timeout-minutes and the main ruleset's merge-queue check_response_timeout_minutes are both 30, while the macOS leg of a source change measures 23 to 30 minutes, so ordinary runner variance cancels a passing run, and nothing warns before the cancellation arrives. Ruling Z (2026-09-28, the technical facilitator) raises both limits to 45 minutes and asks for a warning once the macOS check passes 35 minutes, so a speed lane opens before any cancellation.
- iss-2610090642376032A pull request the implement loop lands can leave the merge queue silently. Twice it sat CLEAN with auto-merge armed and never entered the queue (#837, #844; the enqueue mutation then failed with a forge error and only toggling auto-merge queued it), and four times across this run and the v0.13.2 cut a failed merge-group check (the macOS job cap, or a flaky test) dropped it from the queue (#839, #854, #867, #869) with nothing re-queuing it. The loop arms auto-merge and walks away, so a run that trusts it stalls unseen; one drop cost about 45 minutes in this run.
- iss-2610110938282654check (macos-latest) runs on every pull request as well as on every merge-queue entry, and macOS is the account's scarcest runner, 5 jobs at once at a median 40 minutes each (8 queue runs, 2026-10-11). Between 03:36 and 08:40 on 2026-10-11 nothing merged: entries waiting for a macOS slot passed the queue's timeout and were removed. Every push already passed make preflight on a Mac, so the pull-request macOS run repeats a local run, and only the queue run tests the merged tree.
- itd-115A Ready PR Merges Without Ever Wedging BEHIND
- itd-2610110940216682Every change is checked in full once, on exactly what lands