SWE-Pair · public split

62 public tasks

Bug report, hidden tests, reference fix and every agent attempt for each. 99 more are held out and never listed. Request the full set →

TaskProjectTierf2pMergedDeepSeek V4.1 Flash · maxGPT-6 Luna · maxDeepSeek V4.1 Flash · mediumGPT-5.6 Luna · mediumGLM-5.3 Flash · medium
agronholm-anyio-1279-1223agronholm/anyioclean42026-08-231/11/11/14/41/1
agronholm-anyio-1397-1403agronholm/anyioclean122026-10-101/11/11/12/21/1
aio-libs-aiohttp-8445-8482aio-libs/aiohttpclean42024-07-171/11/11/12/21/1
atlanhq-application-sdk-1705-1744atlanhq/application-sdkclean142026-05-141/11/11/12/21/1
berriai-litellm-16672-16708BerriAI/litellmclean32025-11-181/11/11/12/21/1
berriai-litellm-20666-22884BerriAI/litellmsplit52026-03-050/11/11/12/20/1
berriai-litellm-21658-22884BerriAI/litellmclean32026-03-051/11/11/12/20/1
bumpy-croc-ai-trading-bot-636-637bumpy-croc/ai-trading-botclean52026-06-011/11/11/11/21/1
celery-celery-10108-10098celery/celeryclean52026-02-221/11/11/12/21/1
celery-celery-10179-10189celery/celerysplit192026-03-101/10/11/12/20/1
celery-celery-10189-10158celery/celeryclean152026-04-041/11/11/12/20/1
celery-celery-10218-10204celery/celeryclean62026-03-261/11/11/14/41/1
celery-celery-10408-10404celery/celerysplit42026-07-151/10/10/10/20/1
celery-celery-10750-10834celery/celeryclean72026-10-131/11/11/11/21/1
celery-celery-9815-9853celery/celeryclean92025-08-311/11/11/12/20/1
celery-celery-9815-9863celery/celeryclean132025-08-311/11/11/11/21/1
celery-kombu-2492-2552celery/kombusplit62026-06-101/10/11/11/20/1
celery-kombu-2498-2492celery/kombusplit92026-03-261/10/11/12/21/1
celery-kombu-2498-2492-2552celery/kombusplit102026-06-101/10/10/10/20/1
dewet22-givenergy-hass-51-73dewet22/givenergy-hassclean22026-05-301/11/11/12/41/1
fastapi-fastapi-14349-14297fastapi/fastapiclean32025-12-021/11/11/13/41/1
flyteorg-flyte-sdk-1142-1178flyteorg/flyte-sdkclean52026-06-081/11/11/12/21/1
hkuds-lightrag-3056-3129HKUDS/LightRAGhard152026-05-240/10/11/10/20/1
jd-tenacity-606-609jd/tenacityclean42026-02-241/11/11/12/21/1
jd-tenacity-606-615jd/tenacityclean62026-02-251/11/11/12/21/1
jd-tenacity-615-614jd/tenacityclean82026-02-251/11/11/12/21/1
jetbrains-research-opaque-519-542JetBrains-Research/opaqueclean92026-08-111/11/11/12/20/1
k1rl3s-maxo-345-348K1rL3s/maxosplit112026-09-140/11/11/10/20/1
komalharshita-devpath-1846-1890komalharshita/DevPathhard192026-08-130/10/11/11/20/1
learningcircuit-local-deep-research-2517-2502LearningCircuit/local-deep-researchclean102026-03-061/11/11/12/21/1
microsoft-apm-566-609microsoft/apmhard112026-04-100/10/11/12/21/1
microsoft-apm-596-609microsoft/apmclean122026-04-101/11/11/11/20/1
nithin031-noema-1-2Nithin031/Noemaclean72026-09-131/11/11/12/21/1
nousresearch-hermes-agent-32018-33241NousResearch/hermes-agentclean102026-05-271/11/11/12/21/1
nousresearch-hermes-agent-83536-99176NousResearch/hermes-agentclean92026-08-311/11/11/12/21/1
pandas-dev-pandas-66327-66275pandas-dev/pandasclean42026-07-201/11/11/12/21/1
pandas-dev-pandas-67980-68034pandas-dev/pandasclean222026-09-051/11/11/10/21/1
pandas-dev-pandas-68039-68067pandas-dev/pandasclean62026-09-051/11/11/12/21/1
pandas-dev-pandas-68524-68587pandas-dev/pandashard3252026-09-240/10/1–––
pandas-dev-pandas-69131-68524pandas-dev/pandashard6792026-09-220/10/1–––
pavua-krab-ear-1161-1290Pavua/Krab-Earclean42026-05-271/11/11/14/41/1
pavua-krab-ear-1264-1290Pavua/Krab-Earclean32026-05-271/11/11/12/21/1
pavua-krab-ear-1297-1290Pavua/Krab-Earclean42026-05-271/11/11/14/41/1
prefecthq-fastmcp-3827-3889PrefectHQ/fastmcphard172026-04-140/10/10/11/21/1
prefecthq-fastmcp-3899-3889PrefectHQ/fastmcpclean1292026-04-141/11/11/12/21/1
prefecthq-fastmcp-3925-3889PrefectHQ/fastmcpsplit142026-04-141/10/11/10/20/1
prefecthq-prefect-19601-19708PrefectHQ/prefectclean42025-12-101/11/11/12/21/1
protolabsai-protoagent-302-304protoLabsAI/protoAgentclean42026-05-301/11/11/14/41/1
pydantic-pydantic-ai-4781-4687pydantic/pydantic-aiclean72026-03-311/11/11/11/20/1
pytest-dev-pytest-14540-14567pytest-dev/pytestclean42026-06-091/11/11/10/20/1
pytest-dev-pytest-14566-14540pytest-dev/pytestclean32026-06-081/11/11/12/21/1
rhysu-droll-182-190RhysU/drollclean42026-03-041/11/11/14/41/1
tornadoweb-tornado-3553-3554tornadoweb/tornadoclean22025-12-101/11/11/12/21/1
tox-dev-python-discovery-79-85tox-dev/python-discoverysplit112026-06-110/11/11/12/20/1
tox-dev-sphinx-autodoc-typehints-605-606tox-dev/sphinx-autodoc-typehintsclean22026-02-181/11/11/14/41/1
tox-dev-sphinx-autodoc-typehints-607-605-606tox-dev/sphinx-autodoc-typehintsclean42026-02-181/11/10/11/21/1
tox-dev-sphinx-autodoc-typehints-607-606tox-dev/sphinx-autodoc-typehintsclean32026-02-181/11/11/14/41/1
tox-dev-tox-3825-3838tox-dev/toxclean252026-02-271/11/11/10/21/1
tox-dev-tox-3838-3842tox-dev/toxclean212026-03-011/11/11/10/21/1
vantasnerdan-claude-code-xai-21-22vantasnerdan/claude-code-xaiclean832026-02-221/11/10/12/20/1
vyperlang-vyper-3925-4059vyperlang/vyperclean132024-05-301/11/11/10/20/1
vyperlang-vyper-3925-4060vyperlang/vypersplit172024-06-021/10/10/10/20/1