Development record
2026-08-05 — iss-188 (bug-hunt loop, follow-on to iss-185): a secret pattern whose quantifier is open-ended (`\bghp_[A-Za-z0-9]{36,}` and the seven other `{n,}` families) greedily consumes a following token's own leading bytes when those fall inside its trailing character class, so the true junction sits BEFORE the match's reported end and iss-185's forward adjacency probe — which only ever looks at that end — never runs where the second token actually starts. `"ghp_"+36×a+"ghp_"+36×b` reported one finding covering `ghp_aaa…aaaghp` and left `_bbb…bbb` raw through `Redact`, with `history`'s fail-closed residual re-scan reporting the output clean: the same live-secret-reaches-disk violation iss-185 closed for the fixed-length family. Fixed in `internal/adapter/scanner/scanner.go:465` (`stolenJunctions`): before a match's reported end is accepted as final, a bounded backward search walks candidate cuts and reports each one where the shortened prefix is STILL a whole match for its own pattern AND a token can begin, and `scanAllPatterns` probes those cuts exactly as it probes the forward end. Open-ended is detected from the compiled regexp — a match that cannot still match one byte shorter has a rigid length and is skipped after one test — not from hand-annotated metadata, so a pattern added later inherits the classification. Three things keep the cost per match rather than per match LENGTH, which the ledger entry named as the trap (an unbounded backward scan would trade the leak for a resource-exhaustion cliff on exactly the huge single-line input a scanner must handle): that one-test early-out, `maxAdjacencyBacktrack` (512, mirroring `maxAdjacencyProbeWindow`), and a single combined `junctionProbe` alternation over the whole pattern set that generates candidate cuts in one linear pass instead of one pass per pattern — measured 40s unbounded against 0.3s bounded on a 200KB line. The over-long match is deliberately left untrimmed: it only over-reports the first token's span, and `sealLine` already forces every overlap byte to `*`. Repro: `internal/adapter/scanner/adjacency_test.go`, `TestConcatenatedOpenEndedSecretsBothDetected` (the ledger entry's literal reproducer) and `TestOpenEndedSecretSwallowingDifferentFamilyBothDetected` (a whole `AKIA` key swallowed by an alnum class run), both watched failing on pre-fix code for the claimed reason; cost guard `TestJunctionBacktrackIsBounded`.
2026-08-05 — iss-188 (bug-hunt loop, follow-on to iss-185): a secret pattern whose quantifier is open-ended (\bghp_[A-Za-z0-9]{36,} and the seven other {n,} families) greedily consumes a following token's own leading bytes when those fall inside its trailing character class, so the true junction sits BEFORE the match's reported end and iss-185's forward adjacency probe — which only ever looks at that end — never runs where the second token actually starts. "ghp_"+36×a+"ghp_"+36×b reported one finding covering ghp_aaa…aaaghp and left _bbb…bbb raw through Redact, with history's fail-closed residual re-scan reporting the output clean: the same live-secret-reaches-disk violation iss-185 closed for the fixed-length family. Fixed in internal/adapter/scanner/scanner.go:465 (stolenJunctions): before a match's reported end is accepted as final, a bounded backward search walks candidate cuts and reports each one where the shortened prefix is STILL a whole match for its own pattern AND a token can begin, and scanAllPatterns probes those cuts exactly as it probes the forward end. Open-ended is detected from the compiled regexp — a match that cannot still match one byte shorter has a rigid length and is skipped after one test — not from hand-annotated metadata, so a pattern added later inherits the classification. Three things keep the cost per match rather than per match LENGTH, which the ledger entry named as the trap (an unbounded backward scan would trade the leak for a resource-exhaustion cliff on exactly the huge single-line input a scanner must handle): that one-test early-out, maxAdjacencyBacktrack (512, mirroring maxAdjacencyProbeWindow), and a single combined junctionProbe alternation over the whole pattern set that generates candidate cuts in one linear pass instead of one pass per pattern — measured 40s unbounded against 0.3s bounded on a 200KB line. The over-long match is deliberately left untrimmed: it only over-reports the first token's span, and sealLine already forces every overlap byte to *. Repro: internal/adapter/scanner/adjacency_test.go, TestConcatenatedOpenEndedSecretsBothDetected (the ledger entry's literal reproducer) and TestOpenEndedSecretSwallowingDifferentFamilyBothDetected (a whole AKIA key swallowed by an alnum class run), both watched failing on pre-fix code for the claimed reason; cost guard TestJunctionBacktrackIsBounded.
Frontmatter
| id | dcn-2608050000000010 |
|---|---|
| type | decision |
| date | 2026-08-05 |
.abcd/work/decisions/dcn-2608050000000010-iss-188-bug-hunt-loop-follow-on-to.md
open on GitHub ↗ · commit history ↗
mentions2
- iss-185scanner leaves a second back-to-back secret token undetected and unredacted, and the fail-closed residual re-scan reports clean anyway. Fixed-length token patterns (e.g. github_pat_ at patterns.go:93) require a leading \b; when two same-family tokens are concatenated with no separator, the first regex match consumes bytes up through the second token's first byte, leaving word/word at the junction so the second token's \b can never match. ScanText (scanner.go:274) therefore returns only one finding. Redact's fingerprintSpan keeps 2 raw alnum bytes at the tail of the masked span (scanner.go:393), so the junction stays word/word after redaction too — the second token survives completely raw in the output. Because the boundary never recovers, history.Capture's stage-two residual check (history.go:159, blockingResidual) re-scans the redacted text, finds nothing, and writes the record to disk with a live secret still in it, directly violating history's documented fail-closed guarantee ('a stored record can never contain a live secret'). Distinct from iss-65 (Finding.MarshalJSON snippet cross-leak in serialized findings, already fixed) — this is a ScanText/Redact detection gap in the scan pass itself. Reproducing test: internal/adapter/scanner/, TestConcatenatedSecretsBothDetected.
- iss-188scanner adjacency recovery misses greedy over-consumption in open-ended-quantifier secret patterns. iss-185's fix (scanAllPatterns, scanner.go) recovers a second token that immediately abuts a first with no separator ONLY when the first pattern is fixed-length (its match always ends exactly where the second token begins, so probing every pattern's boundary-free variant at that one byte offset is sufficient). A pattern whose quantifier is open-ended (e.g. `\bghp_[A-Za-z0-9]{36,}`, `\bsk-ant-[A-Za-z0-9_-]{40,}`, `\bsk_live_[A-Za-z0-9]{20,}`) instead greedily consumes as many class-matching bytes as are available, including a following token's own leading bytes when those happen to fall inside the same character class — shifting the true junction earlier than the reported match end, before scanAllPatterns' adjacency probe ever runs. Concrete repro: `"ghp_" + strings.Repeat("a",36) + "ghp_" + strings.Repeat("b",36)` — the first `ghp_` pattern's `{36,}` class run consumes the 36 a's AND the second token's leading "ghp" (all alnum), stopping only at the second token's `_` (not in `[A-Za-z0-9]`); ScanText reports exactly ONE finding covering `ghp_aaaa...aaaghp`, and the second token's `_bbbb...bbb` tail survives Redact completely raw with the fail-closed residual re-scan reporting clean — the same "can never contain a live secret" violation iss-185 was filed for, just for the open-ended-quantifier pattern family instead of the fixed-length one. Distinct from iss-185 (closed): that fix's mechanism (probe every pattern at a match's reported end) is necessary but not sufficient — this needs a bounded backward search (shrink the greedy match's end toward its start, checking at each candidate cut whether the shortened match is still valid for its own pattern AND some pattern's boundary-free variant matches there) without becoming O(line length) per match, since these patterns' matches can legitimately be very long (base64 blobs, minified files) and an unbounded backward scan across every byte would be a new resource-exhaustion risk on exactly the kind of large single-line input a secret scanner must handle. Surfaced by an adversarial pre-PR security review of iss-185's fix (bug-hunt loop round 3), not by the round's own hunt; not fixed in that PR to keep its diff scoped to the exact bug iss-185 named and to avoid shipping an unbounded-cost search under review pressure. Affected pattern families: github_pat (ghp_/ghs_/gho_/ghu_/ghr_), anthropic, openai_project, openai_svcacct, stripe_live, stripe_test, slack, jwt_shaped — every bundled pattern with an open (`{n,}`) rather than fixed (`{n}`) quantifier.