What it is
SWE-smith makes software engineering tasks by putting bugs into real Python repositories. Each task is a branch of a mirror repository with the bug in place and the tests that catch it removed; the check brings those tests back and runs them.
Ready to run
A run passes when the tests the bug broke pass again. This sample task was checked: its check fails on the starting code and passes with the known fix. Run it from Prepare eval in the BoltzLabs desktop app.
| Task | PyPika: joins on UPDATE queries and self-joins In PyPika, QueryBuilder.do_join (pypika/queries.py) has regressed. Joining a table onto an UPDATE query, such as Query.update(t).join(t2).on(...), now raises JoinException "Invalid join criterion", and joining the same table as the FROM table without an alias no longer aliases it (expected JOIN "abc" "abc2"). Fix do_join so both work again, without changing the tests. |
|---|---|
| Repository | github.com/swesmith/kayak__pypika.1c9646f0 at kayak__pypika.1c9646f0.lm_rewrite__1fkaem5n |
| Check | git fetch -q --depth 1 origin main && git checkout -q FETCH_HEAD -- pypika/tests/dialects/test_postgresql.py pypika/tests/test_joins.py pypika/tests/test_updates.py && python3 -m pytest -q pypika/tests/dialects/test_postgresql.py pypika/tests/test_joins.py pypika/tests/test_updates.py |
| Time limit | 180 seconds |
Use it with BoltzLabs
BoltzLabs does not host this project. To record an agent working on one of its tasks:
- Prepare one task from the project: its starter files, instructions, dependencies and check command.
- In Evals, choose New eval, upload the starter folder or enter its repository, then set the task and check command.
- Choose BoltzLabs models or your own endpoint. Each run gets its own machine.
- When the run finishes, open it to read the agent’s trace, file changes and check result.
Projects that need a desktop or phone, self-hosted websites, a game or a simulator need that runtime set up separately; an eval machine does not include it. Machine time uses account credits; hosted model calls are billed separately.