Changelog
What changed
Currentv0.1updated 3 Oct 2026
Tasks18895 public · 93 private
Models3DeepSeek V4 Flash, GLM-5.3 Flash, GPT-5.6 Luna
Entries4since 25 Sep 2026
v0.14 entries · 25 Sep 2026 → 3 Oct 2026
3 Oct 2026
DeepSeek V4 Flash added (387 attempts). The leaderboard now counts every attempt made against the shipped version of a task, including the September calibration runs: GLM-5.3 Flash 441 attempts, GPT-5.6 Luna 1,178. GLM moves from 85.1% (one attempt per task) to 81.9%; ranks are unchanged.
2 Oct 2026
First results: GPT-5.6 Luna (4 attempts per task) and GLM-5.3 Flash (1 attempt). 188 tasks after a release check that excluded 3; see the methodology page.
29 Sep 2026
Public/private split regrouped so that tasks sharing an upstream pull request stay in one half (21 pull requests had crossed). 95 public and 95 private tasks before verification.
25 Sep 2026
Pool frozen at 191 tasks from 108 projects. Copyleft, unlicensed and noncommercial projects excluded.