Batching related judge items into one request beats one request per item on a local OpenAI-compatible model service. A sibling session's measurement on 2026-10-03 (44 labelled lines of banned-term hits and near-misses, one local 8-bit model, streaming, temperature 0, max_tokens from the served context): all 44 in one request scored 41/44 in 181 s and 8,872 completion tokens; one request per item scored 30/44 in 2,487 s (14x) and 116,044 completion tokens (13x). One-by-one lost cross-item context (five false blocks on abcd's own documentation lines, two misses), let reasoning run away (three single items ran 566-589 s and hit max_tokens with no verdict), and paid a fixed overhead per request. abcd's delegated many-item steps (consistency, cold readings) are batch-shaped, so fanning them out per item to a local provider without their context is the losing shape. More data from two further 27B models in both modes is to follow.

Batching related judge items into one request beats one request per item on a local OpenAI-compatible model service. A sibling session's measurement on 2026-10-03 (44 labelled lines of banned-term hits and near-misses, one local 8-bit model, streaming, temperature 0, max_tokens from the served context): all 44 in one request scored 41/44 in 181 s and 8,872 completion tokens; one request per item scored 30/44 in 2,487 s (14x) and 116,044 completion tokens (13x). One-by-one lost cross-item context (five false blocks on abcd's own documentation lines, two misses), let reasoning run away (three single items ran 566-589 s and hit max_tokens with no verdict), and paid a fixed overhead per request. abcd's delegated many-item steps (consistency, cold readings) are batch-shaped, so fanning them out per item to a local provider without their context is the losing shape. More data from two further 27B models in both modes is to follow.

Evidence 2026-10-03 (second relay from the same sibling session)

The same 44-line test, streaming, temperature 0, max_tokens from the served context, on a local OpenAI-compatible service (unnamed here), across five models:

What it adds to the remedy: whether to batch depends on the model (the stronger model gained from cross-item context, the 2-bit model lost track of a long batch), so a batch is checked for completeness, every item answered, and a missing or failed item is re-sent alone with its context; a batch can end normally with fewer answers than items, and the sibling's own scorer first miscounted until unanswered items counted as misses; reasoning models may not converge on a long batch within an hour, so bulk judging prefers non-reasoning or decision models, or bounds reasoning per item; a decision model is a different cost class (about 14 times faster than the fastest chat judge here, no positives missed, but 10 to 13 false positives, and moving its cut-off traded errors about one for one); streaming is required for long runs (the adapter's streaming fix, captured the same day), and in one case a dropped stream left the server generating for minutes.