Development record
The scanner's glued-token sweep (glued.go, iss-2609290541525428) runs on the raw line only: decodedLineFindings and viewFindings (percent.go) scan the percent-decoded and JSON-unescaped views with the bounded patterns alone, whose leading word boundary cannot hold behind a word byte. A glued token whose own bytes are escaped (a percent-encoded letter of its prefix, or a JSON unicode escape of its first byte, behind an underscore or a letter) therefore survives ScanText, Redact and RedactRefusal raw. Reach is narrow, since no encoder escapes an ASCII letter of a token, but the decoded layers exist to close spelling variants. A second half: ScanText gives no signal when the sweep cannot be built in full (a configured pattern whose leading boundary carries a quantifier); only RedactRefusal reads the sweep's completeness, so every other consumer runs a silently narrower sweep, and Unavailable does not say so.
The scanner's glued-token sweep (glued.go, iss-2609290541525428) runs on the raw line only: decodedLineFindings and viewFindings (percent.go) scan the percent-decoded and JSON-unescaped views with the bounded patterns alone, whose leading word boundary cannot hold behind a word byte. A glued token whose own bytes are escaped (a percent-encoded letter of its prefix, or a JSON unicode escape of its first byte, behind an underscore or a letter) therefore survives ScanText, Redact and RedactRefusal raw. Reach is narrow, since no encoder escapes an ASCII letter of a token, but the decoded layers exist to close spelling variants. A second half: ScanText gives no signal when the sweep cannot be built in full (a configured pattern whose leading boundary carries a quantifier); only RedactRefusal reads the sweep's completeness, so every other consumer runs a silently narrower sweep, and Unavailable does not say so.
Grounds
- pursued: ScanText reports, and Redact and RedactRefusal seal, a glued token whose own bytes are percent- or JSON-escaped (TestScanTextFindsAnEscapedGluedToken, TestRedactRefusalSealsAnEscapedGluedToken, eight subtests RED before aa5ea012d), and a configured pattern with a quantified leading boundary makes Unavailable true with its name (TestUnavailableNamesAnIncompleteGluedSweep, RED before 0d3156d28); an escaped glued spelling that still comes back raw, a decoded-view charge growing past the 6.0x bar in TestGluedSweepWorkIsLinear, or a degraded sweep that leaves Unavailable false would show it wrong.
Frontmatter
| id | iss-2609290743362554 |
|---|---|
| type | issue |
| lifecycle | resolved |
| severity | minor |
.abcd/work/issues/resolved/iss-2609290743362554-the-scanner-s-glued-token-sweep-glued-go.md
open on GitHub ↗ · commit history ↗
mentions1
- iss-2609290541525428scanner.RedactRefusal does not seal a token glued behind a word character. The scanner's bundled secret patterns anchor their start on a leading word boundary, and '_' and every letter or digit are word characters, so a token that sits right after one has no boundary and is never matched: the implement loop's laneFileGap (internal/core/implement/loop/receipt.go) returns the receipt's path as notes_ followed by a GitHub PAT and .md, does not exist, with the token raw, and memory's ValidateDistilledPage (internal/core/memory/schema.go) returns unknown key(s) [notes_ followed by the token] raw, while notes- and sub/ spellings seal. A letter on both sides (x, token, y) is missed the same way. Every RedactRefusal caller shares the gap (scribe, release, ideate, lifeboat, reading, intent, memory, implement loop), so a host payload's key or path carries a credential to the terminal and the transcript through a refusal. It is the refusal-side twin of the page-filename gap iss-2609290411321963 closed with filenameJudgeTexts. Found by the security review of lane drainEcho3. Fix direction: in the primitive, not the callers; re-find the secret patterns with their leading boundary removed, through the scanner's own linear adjacency machinery, and seal every glued match byte for byte through Redact; fail closed as before. Detector: a refusal naming a key or a path in which a well-formed token sits behind an underscore or a letter carries the token sealed.