You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Label SWE-bench walkthrough as projection, not measured result
The ~85% figure is a target trajectory; make that unmistakable in the
section title, intro note, headline, and chart annotation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: README.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -121,11 +121,11 @@ Every attempt is written to a **ledger**: goal, observation, plan, diff, evaluat
121
121
122
122
---
123
123
124
-
## A worked example: SWE-bench Verified
124
+
## A projected walkthrough: SWE-bench Verified
125
125
126
-
> 📋 The walk-through below uses an end-to-end overnight evolution on SWE-bench Verified to illustrate what the runtime actually does: which model goes in, what kinds of moves the planner converges on, what the final ledger looks like. Reproducible v1.0 artifacts live in [`examples/sandbox_demo/`](examples/sandbox_demo/) and the 99-test suite in [`tests/`](tests/).
126
+
> 📋 The walk-through below sketches an end-to-end overnight evolution on SWE-bench Verified to illustrate what the runtime actually does: which model goes in, what kinds of moves the planner converges on, what the final ledger looks like.**The ~85% figure below is a projected target, not a measured result.** Reproducible v1.0 artifacts live in [`examples/sandbox_demo/`](examples/sandbox_demo/) and the 99-test suite in [`tests/`](tests/).
127
127
128
-
### Take Qwen3.6-35B-A3B (3B active params, released April 2026) from 73.4% to ~85% on SWE-bench Verified — closing most of the gap to GPT-5.5, overnight, hands-off, fully audited.
128
+
### Target: take Qwen3.6-35B-A3B (3B active params, released April 2026) from 73.4% to ~85% on SWE-bench Verified — closing most of the gap to GPT-5.5, overnight, hands-off, fully audited.
129
129
130
130
```
131
131
SWE-bench Verified (500 real GitHub bug-fixes · May 2026)
@@ -134,7 +134,7 @@ Every attempt is written to a **ledger**: goal, observation, plan, diff, evaluat
134
134
Gemini 3.1 Pro ████████████████████████████████░░░░░ 80.6%
135
135
Kimi K2.6 ████████████████████████████████░░░░░ 80.2%
0 commit comments