Development record
2026-07-14 (research, UNVERIFIED): the SWE-bench Pro audit figures (27.4% automated /
decisiondcn-2607140000000015
2026-07-14 (research, UNVERIFIED): the SWE-bench Pro audit figures (27.4% automated / 34.1% human) and the SWE-bench Verified figures (>=59.4% flawed tests / 35.5% hidden implementation details) come from SECONDARY sources — openai.com 403s to the research fetcher. These numbers are the reference standard the proposed paper is scored against. VERIFY AGAINST THE PRIMARY before writing anything. Do not cite the decimals until read first-hand.
Frontmatter
| id | dcn-2607140000000015 |
|---|---|
| type | decision |
| date | 2026-07-14 |
.abcd/work/decisions/dcn-2607140000000015-the-swe-bench-pro-audit-figures-27-4.md
open on GitHub ↗ · commit history ↗