Development record
abcd must never do this to a machine when running tests for a repo it manages. On 2026-09-21 at 08:16Z the development Mac kernel-panicked (userspace watchdog: no successful check-ins from WindowServer in 120 seconds, two induced crashes, then the panic), killing the pilot run's orchestrator session mid-gate and losing its scratchpad. The stackshot taken 25 seconds before the first WindowServer kill shows CPU starvation, not memory: eight orphaned /usr/bin/yes processes pegging eight cores for 2.3 days, two spinning zsh loops, and abcd's own reading.test and cli.test running under them from another session's go test; the OS's low-priority daemons (dasd at priority 4) got under a millisecond of CPU, 500-plus threads queued behind them through turnstiles, and WindowServer blocked in a synchronous XPC to cfprefsd that was itself blocked behind dasd. The burners came from a test-under-load experiment in a managed repository on 2026-09-19: an agent launched eight of them with nohup yes and appended each pid to a file whose directory did not exist yet, so the launches succeeded and the bookkeeping failed; the later kill loop read the pid file, pgrep -x yes still printed 8, a second kill pass over the same file followed, and the agent declared the machine clean. The pattern is one abcd can be asked to run and must refuse to run this way. Wanted, as the rule the record carries: (1) a test or experiment that spawns background load owns its children in a way that survives a failed bookkeeping step, a process group or setsid with the group killed, or a parent that dies with the session, never nohup with pids in a file as the only handle; (2) clean is proven by a process query, pgrep -x on the session's own burner name returning nothing, never by the pid file having been read; (3) a load experiment on a live development machine needs explicit consent and a hard ceiling below the core count, leaving cores for the OS's own daemons, and abcd's own test run is never started while such load is present, so the gate reads the load average before it begins and refuses above a declared ceiling, naming what is burning. Severity major: it took the product thinker's machine down and lost a session. Diagnostic report names (under the system's diagnostic-reports directory): panic-full-2026-09-21-091803.0002.panic, WindowServer-2026-09-21-091552.ips, WindowServer-2026-09-21-091632.ips, and two WindowServer userspace_watchdog_timeout spin stackshots.
abcd must never do this to a machine when running tests for a repo it manages. On 2026-09-21 at 08:16Z the development Mac kernel-panicked (userspace watchdog: no successful check-ins from WindowServer in 120 seconds, two induced crashes, then the panic), killing the pilot run's orchestrator session mid-gate and losing its scratchpad. The stackshot taken 25 seconds before the first WindowServer kill shows CPU starvation, not memory: eight orphaned /usr/bin/yes processes pegging eight cores for 2.3 days, two spinning zsh loops, and abcd's own reading.test and cli.test running under them from another session's go test; the OS's low-priority daemons (dasd at priority 4) got under a millisecond of CPU, 500-plus threads queued behind them through turnstiles, and WindowServer blocked in a synchronous XPC to cfprefsd that was itself blocked behind dasd. The burners came from a test-under-load experiment in a managed repository on 2026-09-19: an agent launched eight of them with nohup yes and appended each pid to a file whose directory did not exist yet, so the launches succeeded and the bookkeeping failed; the later kill loop read the pid file, pgrep -x yes still printed 8, a second kill pass over the same file followed, and the agent declared the machine clean. The pattern is one abcd can be asked to run and must refuse to run this way. Wanted, as the rule the record carries: (1) a test or experiment that spawns background load owns its children in a way that survives a failed bookkeeping step, a process group or setsid with the group killed, or a parent that dies with the session, never nohup with pids in a file as the only handle; (2) clean is proven by a process query, pgrep -x on the session's own burner name returning nothing, never by the pid file having been read; (3) a load experiment on a live development machine needs explicit consent and a hard ceiling below the core count, leaving cores for the OS's own daemons, and abcd's own test run is never started while such load is present, so the gate reads the load average before it begins and refuses above a declared ceiling, naming what is burning. Severity major: it took the product thinker's machine down and lost a session. Diagnostic report names (under the system's diagnostic-reports directory): panic-full-2026-09-21-091803.0002.panic, WindowServer-2026-09-21-091552.ips, WindowServer-2026-09-21-091632.ips, and two WindowServer userspace_watchdog_timeout spin stackshots.
Grounds
- pursued: an agent about to run background load in a managed repository is handed the three rules and a working group-kill idiom; a load experiment that still orphans processes after the domain injected, or a test showing the idiom leaves a grandchild running, would show it wrong
Frontmatter
| id | iss-2609210828122412 |
|---|---|
| type | issue |
| lifecycle | resolved |
| severity | major |
.abcd/work/issues/resolved/iss-2609210828122412-a-test-that-spawns-background-load-must.md
open on GitHub ↗ · commit history ↗
References1
- itd-2609231434459890abcd's own tests warn before running on an overloaded machinerelated
mentions4
- dcn-26092300000000132026-09-23 — A load experiment is trusted only under four conditions (product thinker, ruling on iss-2609210828122412, after orphaned load processes from another repository starved the machine into a crash): background load processes start as one owned process group and are killed together; "clean" is proven by checking what is actually running, never by reading a list of what was started; an experiment on a live development machine needs explicit consent and a cap below the core count; and abcd's own test lanes check machine load before they start and refuse when it is too high, naming what is running. The first three hold as rules now; the last is an intent to plan.
- dcn-26092300000000212026-09-23 — The load check before abcd's own test lanes warns and never refuses (product thinker, planning interview for itd-2609231434459890; spc-2609231542463113, Decision 2). This changes the fourth condition of the run A interview ruling on iss-2609210828122412, which had the check refusing. The check names foreign sustained CPU (a near-full core for longer than 30 minutes) and an extreme load (four times the online cores), both limits overridable in the caller's `~/.abcd/load-limits`; it names the caller's own strays with a remedy and counts other accounts' processes without naming them; it writes a run-log event; it runs once at the start of `make preflight` and of the eval harness; CI skips it with a stated reason; it always exits 0, and its findings are minor.
- iss-2609240646538696The shell guard's bundled hazard registry has no entry for a kill by pattern: `abcd guard check --command "pkill -f 'make preflight'"` answers allow, and so does `killall make`. On 2026-09-23 a lane agent in autonomous run A stopped its own gate with `pkill -f "make preflight"` on a machine where several sessions ran gates at once. The pattern matched every session's preflight, so two peer lanes lost theirs, and the loss read as an unexplained SIGTERM for about half an hour until the agent's command was found. The run's lane brief then forbade pattern kills, which is the discipline rung. Wanted: a registry entry that blocks, or at least warns on, `pkill -f`, `pkill` and `killall` with a name or pattern, and a kill whose pid list comes from a pattern search, naming the successor: kill the pid you recorded when you started the process, or its own process group. The LOAD rule domain (iss-2609210828122412) already states the same rule for a test's own children.
- spc-2609231542463113The load check: preflight and the eval harness warn about strays and extreme load, and never refuse