The load gate's queue orders waiters by pid, so a waiter with a high pid starves once the machine's process ids wrap. machineload.QueueAhead counts another waiter as ahead when its pid is lower than this one's. After a reboot on 2026-10-16 the ids wrapped: waiters started between 16:07 and 17:26, with pids 50460 to 71218, all queued ahead of one started at 16:04 with pid 96274. That waiter could only wait out its timeout while lower pids kept arriving. The docs and the LOAD rule promise one waiter at a time in queue order.

Frontmatter

.abcd/work/issues/resolved/iss-2610161634043273-the-load-gate-s-queue-orders-waiters-by.md
open on GitHub ↗ · commit history ↗

References1

• itd-2610170634394993Headless agents in parallel coordinate through the shared remote, not conversationrelated

mentions4

• adr-2610222026262980Coordination between headless agents is bounded by deadlines, fenced by the claim's tip, version-checked, and done under each session's own credential
• dcn-26101622393089032026-10-16 — The load gate's queue (iss-2610161634043273) counts a queued waiter as ahead only when it is ahead by start time and also has the lower pid. Rejected alternatives: start order alone, as the branch had it, deadlocks against waiters built from code that predates the fix, which wait on every lower pid; on the night of 2026-10-16 a young low-pid waiter and two older high-pid ones each saw the other ahead for hours. Reverting to pid order alone also meets the no-mutual-wait property, but brings back the starvation after a pid wrap that the issue fixed. Telling old waiters from new ones in the process table is not possible, since both run the same arguments, so any safe rule may only wait on lower pids. Within that limit, start order AND lower pid keeps the most of the fix, and it is the smallest change: queueOrder and its chained-tie grouping are unchanged, and the pid check is one condition on top. Every wait from either code now points at a lower pid, so no cycle of any length can form, whatever mix of code and whatever each waiter's own read of the table. The grouping no longer prevents cycles; it only fixes the order within a chain of ties. After a wrap, an early high-pid waiter is held only behind waiters that started before it. Two waiters whose start and pid disagree wait on neither. Within one account the flock-serialised hold keeps them from opening together. Across accounts there is no shared hold: only the settle jitter of 15 to 60 seconds and the re-check keep them apart, and a re-check that reads the table in the moment between the other's open and its command showing in the table can still open. That gap is narrow and rare, since it needs a pid wrap, two accounts and coinciding re-checks. A pair in pid order has no such gap. It is accepted rather than closed with a cross-account file, which iss-2610140638074587 rules out.
• dcn-26101622574775712026-10-16 — Refines dcn-2610162239308903, the load gate's queue (iss-2610161634043273), on the gap it accepted and on how far its no-cycle claim reaches. The gap is larger than that record says. Two waiters whose start and pid disagree, as after the process ids wrap, wait on neither, and across accounts no hold keeps them apart, so re-checks that fall in one moment, before either shows in the process table as running its command, both open, and their commands overlap. Under `--wait -- make preflight` the overlap becomes a stall: each preflight's own gate, the inner waiter that takes its running ancestor's place, still counts every other waiter running its command as ahead, so each waits on the other's outer waiter and both stand until their two-hour timeouts. It needs a pid wrap, two accounts and coinciding re-checks. The mitigation uses no shared file, which iss-2610140638074587 still rules out. The command form shows itself in the table as running its command from the moment it opens, by a short-lived child of the waiter (a /bin/sleep, its mark, stopped once the command has started), waits two seconds (WaitOpenPause) and reads the table once more before it starts its command. If another waiter is then running its command, opened within that pause and the start tolerance, and has the lower pid, this waiter stands back: it stops its mark, gives up its hold, re-draws its jitter and waits again, while the lower pid keeps going. Because each opener is marked from the moment it opens, two that open at the same instant each see the other, and exactly one runs. A waiter stands back at most three times (WaitBackoffCap) and then opens without the look, so no wait goes round for ever. The plain form starts no command and neither marks nor looks. Within one account nothing changes: the hold already admits one. What stays open: a mark that cannot be started, a table that cannot be read at the look, the cap, and a process start slower than the pause. Rejected: having only the lower pid look, since an opener cannot tell that it is the lower of a disagreeing pair; and closing the stall in the inner gate alone, which leaves two plain race runs overlapping. The earlier record's 'whatever mix of code' covers this code and the pid-only waiters built before the start-order change, and nothing else. A waiter still running the branch's first version, which ordered the queue by start alone, waits on a higher pid that started earlier and can deadlock against pid-only waiters, and no change here can break that, since that waiter never runs it. So when this lands, any waiter running that start-only build is stopped and requeued.
• iss-2610222125175640After macOS process ids wrap, the load gate's queue is no longer first-come-first-served and an old waiter can be starved by every new starter. On 2026-10-22 waiters started 19 s and 9 min earlier held pids 8034 and 12051 while hour-old waiters held 58628 and 73003. The fix for iss-2610161634043273 orders the queue by start time but counts a queued waiter as ahead only when it is ahead by start AND has the lower pid (internal/core/machineload/queue.go, queueOrder/queuedAhead), so after a wrap neither waits on the other; the open-moment stand-back in implement.WaitLoad (openLook, internal/core/implement/load_wait.go ~546) then yields to the lower pid, so the hour-old waiter stands back for each young low-pid starter until its backoffs run out or its timeout (8 h) ends, and a release-blocking preflight can be held all night.