Development record
Raising the single-line PEM opener bar from sixteen characters to forty removed coverage that the previous bar had. An open block, one carrying no END marker, whose visible body is a single line between sixteen and thirty-nine characters followed by any non-body line is no longer redacted, where it was before. The reachable shape is a real one rather than a contrived one: a private key re-wrapped to a narrow column by a mail client or a terminal, or truncated by a log rotation, leaves a header and one short line, and MIIEowIBAAKCAQEA, the canonical opening bytes of a DER-encoded RSA key, then survives verbatim. The failure is silent in the worst way, because the stage-two rescan through BlockingResidual reports no findings and the header pattern still fires, so the store records that a PEM secret was handled while a fragment of the key remains in the text. The argument the change rested on, that no canonically rendered body line falls under forty characters because RFC 7468 mandates sixty-four and OpenSSH writes seventy, is true only of an untruncated and un-rewrapped block, and a truncated or narrow-rewrapped paste is precisely the shape a transcript redactor exists to catch. Three independent adversarial reviews found this, one grading it critical. The fix must restore the lost shape without restoring the defect the raised bar was closing, which was a mentioned header opening a block on any sixteen-character word: the pair rule already distinguishes a wrapped body from prose, so the single-line path needs a discriminator that is not width alone. Detector: a header followed by one body line of thirty-two base64 characters and then a prose line must redact that body line, the two-consecutive-narrow-lines case must keep redacting, and the mentioned-header-plus-fenced-identifier case must still be left untouched.
Raising the single-line PEM opener bar from sixteen characters to forty removed coverage that the previous bar had. An open block, one carrying no END marker, whose visible body is a single line between sixteen and thirty-nine characters followed by any non-body line is no longer redacted, where it was before. The reachable shape is a real one rather than a contrived one: a private key re-wrapped to a narrow column by a mail client or a terminal, or truncated by a log rotation, leaves a header and one short line, and MIIEowIBAAKCAQEA, the canonical opening bytes of a DER-encoded RSA key, then survives verbatim. The failure is silent in the worst way, because the stage-two rescan through BlockingResidual reports no findings and the header pattern still fires, so the store records that a PEM secret was handled while a fragment of the key remains in the text. The argument the change rested on, that no canonically rendered body line falls under forty characters because RFC 7468 mandates sixty-four and OpenSSH writes seventy, is true only of an untruncated and un-rewrapped block, and a truncated or narrow-rewrapped paste is precisely the shape a transcript redactor exists to catch. Three independent adversarial reviews found this, one grading it critical. The fix must restore the lost shape without restoring the defect the raised bar was closing, which was a mentioned header opening a block on any sixteen-character word: the pair rule already distinguishes a wrapped body from prose, so the single-line path needs a discriminator that is not width alone. Detector: a header followed by one body line of thirty-two base64 characters and then a prose line must redact that body line, the two-consecutive-narrow-lines case must keep redacting, and the mentioned-header-plus-fenced-identifier case must still be left untouched.
Grounds
- pursued: the reported shapes redact again while every prose case the raised bar protects stays byte-identical; a prose line consumed by the narrow rule, or a truncated body still surviving under a pasted header, would show it wrong
Frontmatter
| id | iss-2609090932441377 |
|---|---|
| type | issue |
| lifecycle | resolved |
| severity | major |
.abcd/work/issues/resolved/iss-2609090932441377-the-raised-pem-opener-bar-drops-a.md
open on GitHub ↗ · commit history ↗
mentions1
- iss-2609020127210042The PEM block consumer redacts the common renderings of a key body but not every one, and one armour form is not detected at all. scanner.pemBodyLineRe accepts an optional gutter then a single base64 run, so a body rendered as log-prefixed lines (a timestamp/level prefix), as CSV cells, as one element per XML tag, with a trailing hash comment, as a source-string concatenation, as a JSON array on one line, or with double-escaped backslash-n separators is not body-shaped, the block ends at that line and the rest is stored verbatim; the tail of a block longer than maxPEMBodyLines survives by the same route. The RFC4716 armour (four dashes, BEGIN SSH2 ENCRYPTED PRIVATE KEY) does not match the bundled pem_private_key pattern at all, so no block is ever opened for it. Evidence: scanner.pemBodyLineRe and maxPEMBodyLines in internal/adapter/scanner/pem.go, DefaultPatterns pem_private_key in patterns.go. A fix must either widen the body-shape rule over these renderings or arm an entropy rule for a headerless base64 run (iss-96), and add the RFC4716 armour to the header pattern.