The scanner's rigid/open-ended heuristic sends every IPv4 and IPv6 match through stolenJunctions' backward search, a measurable cost regression on ordinary log content. stolenJunctions (internal/adapter/scanner/scanner.go) skips its bounded backward search only for patterns whose match cannot be one byte shorter; net_ipv4 and net_ipv6 (internal/adapter/scanner/network.go) are variable-length, so every address they match in ordinary content enters the search — no custom config, no compile-failure fallback needed (that latter, narrower mechanism remains iss-191's scope and still does not trigger on the bundled set, whose combined alternation compiles cleanly). Measured on identical content, current main (bddb6fa) vs the tree immediately before the iss-188 machinery (eb61acd), 100-line ~12KB documents, all identifiers reserved documentation values (RFC 3849/5737/7042): neutral content 1.0x; one MAC per line 3.6x slower (net_mac itself classifies rigid, but ipv6Re also matches MAC-shaped colon-hex and that overlapping match is open-ended); one compressed IPv6 per line 1.9x; one MD5-style SSH fingerprint per line 23x; one dense colon-hex known_hosts-style line 27x. Because Skip/SkipAt reserved-range filtering runs AFTER scanAllPatterns, reserved addresses that are never reported still pay the full search cost. The iss-188 review also demonstrated a correctness angle on this same path: duplicate hard_fail findings manufactured for a single compressed IPv6, reproduced on this repo's own tracked fixtures (internal/core/memory/memory_test.go, internal/urlguard/urlguard_test.go). Candidate directions: exempt network-class patterns from the backward search where a stolen-junction leak is not plausible for them, or short-circuit when the only patterns whose bodies can match in the backtrack window are network-class; whether the residual cost is then a documented trade-off is a maintainer call. Split out of iss-191 per maintainer ruling (2026-08-07); benchmark method and full table recorded here because the scratch artefacts under .abcd/.work.local/ are ephemeral.

Frontmatter

.abcd/work/issues/resolved/iss-195-scanner-openended-heuristic-cost.md
open on GitHub ↗ · commit history ↗

mentions5

• dcn-26080700000000022026-08-07 — iss-191 re-scoped by maintainer ruling: the round-3 merge-gate
• dcn-26080800000000022026-08-08 — bug-hunt loop round 9 (state issue #197): a multi-angle sweep (5
• dcn-26081500000000012026-08-15 — three forward plans adopted at a maintainer grill, absorbing the existing queue rather than adding to it: `2026-08-15-plugin-user-safety.md` (execution priority 1; owns install-experience Cut B pick-up and the v0.5.0 security remainder; fix queue opens iss-200 then iss-195), `2026-08-15-predictable-development.md` (priority 2; itd-109 promotion is the headline; absorbs the v0.5.0 consistency remainder), `2026-08-15-facilitator-experience.md` (continuous, rides along; supersedes the explainability plan's pick-up role). The plan 1/3 boundary is touch-vs-machinery: what a facilitator directly invokes or reads versus automation that guarantees behaviour without human action. Each plan admits items by a stated admission test with the ledger remaining the backlog of record. Same grill: iss-189/190 retired by SUPERSESSION, not wontfix — the defects are real, the local-patch avenue is exhausted (three BLOCKed rounds), so both resolve into the new iss-229 (the recorded 2026-08-08 galloping-probe/trueMatchEnd structural design) with their repros as its acceptance corpus; and itd-111's one-tap micro-prompt open question graduated to its own capture iss-230, homed in the facilitator plan. Rejected: wontfixing 189/190 (asserts the defects don't matter), a full 104-issue triage into the plans (stale mirror of the ledger), and executing the plans in the listed 1-2-3 order (the safety floor and the compounding accelerators come first).
• iss-188scanner adjacency recovery misses greedy over-consumption in open-ended-quantifier secret patterns. iss-185's fix (scanAllPatterns, scanner.go) recovers a second token that immediately abuts a first with no separator ONLY when the first pattern is fixed-length (its match always ends exactly where the second token begins, so probing every pattern's boundary-free variant at that one byte offset is sufficient). A pattern whose quantifier is open-ended (e.g. `\bghp_[A-Za-z0-9]{36,}`, `\bsk-ant-[A-Za-z0-9_-]{40,}`, `\bsk_live_[A-Za-z0-9]{20,}`) instead greedily consumes as many class-matching bytes as are available, including a following token's own leading bytes when those happen to fall inside the same character class — shifting the true junction earlier than the reported match end, before scanAllPatterns' adjacency probe ever runs. Concrete repro: `"ghp_" + strings.Repeat("a",36) + "ghp_" + strings.Repeat("b",36)` — the first `ghp_` pattern's `{36,}` class run consumes the 36 a's AND the second token's leading "ghp" (all alnum), stopping only at the second token's `_` (not in `[A-Za-z0-9]`); ScanText reports exactly ONE finding covering `ghp_aaaa...aaaghp`, and the second token's `_bbbb...bbb` tail survives Redact completely raw with the fail-closed residual re-scan reporting clean — the same "can never contain a live secret" violation iss-185 was filed for, just for the open-ended-quantifier pattern family instead of the fixed-length one. Distinct from iss-185 (closed): that fix's mechanism (probe every pattern at a match's reported end) is necessary but not sufficient — this needs a bounded backward search (shrink the greedy match's end toward its start, checking at each candidate cut whether the shortened match is still valid for its own pattern AND some pattern's boundary-free variant matches there) without becoming O(line length) per match, since these patterns' matches can legitimately be very long (base64 blobs, minified files) and an unbounded backward scan across every byte would be a new resource-exhaustion risk on exactly the kind of large single-line input a secret scanner must handle. Surfaced by an adversarial pre-PR security review of iss-185's fix (bug-hunt loop round 3), not by the round's own hunt; not fixed in that PR to keep its diff scoped to the exact bug iss-185 named and to avoid shipping an unbounded-cost search under review pressure. Affected pattern families: github_pat (ghp_/ghs_/gho_/ghu_/ghr_), anthropic, openai_project, openai_svcacct, stripe_live, stripe_test, slack, jwt_shaped — every bundled pattern with an open (`{n,}`) rather than fixed (`{n}`) quantifier.
• iss-191junctionProbe's compile fallback turns stolenJunctions from a bounded candidate walk into a per-byte one, a latent cost cliff. junctionProbe (internal/adapter/scanner/scanner.go) builds one combined alternation over every pattern's boundary-free body; if that alternation fails to compile it falls back to regexp.MustCompile(`(?s).`), which matches at EVERY byte offset. The fallback was chosen so the candidate generator can over-produce but never under-produce — correct for detection, but it changes stolenJunctions' cost class: instead of one wholeMatch validation per real candidate junction in the backtrack window, wholeMatch runs once per byte offset in the window, and each of those validations is itself proportional to the match's own length (it re-runs the anchored probe over line[m.start:cut]). Cost per match goes from O(window) validations to O(window x match length) work. A reviewer measured roughly a 500x slowdown substituting the fallback directly — 4.21s against 8.25ms on one 200KB match. Not reachable with the current bundled pattern set: every bundled body alternates cleanly, and triggering the compile failure needs a pathological custom .abcd/config/pii.json override whose added pattern makes the joined alternation invalid (the per-pattern regexes are each compiled and validated on merge, so this is hard but not provably impossible to reach). Latent rather than urgent. Options if it is ever worth closing: cap the fallback's candidate count the way the window caps the search, drop patterns from the alternation one at a time until it compiles rather than abandoning the whole set, or mark the scanner unavailable (fail-closed) when the combined probe cannot be built — the last is consistent with how the package already treats a config fault.