What it is
We write jobs an engineer on your team would do in a shell, such as fixing a broken build, running a migration or changing a configuration. Each starts from a prepared state of your repository and ends with a script that checks the result.
What you give us
Your repository, the commands your team uses to build and run it, and the kinds of jobs to cover.
How a run is checked
A run passes when the task’s check script finds the expected files, output or state.
Request it
We prepare this eval from your repository; there is nothing to set up yourself. Request it from Prepare eval in the BoltzLabs desktop app: choose Terminal tasks, add your code and say what the eval should measure. We reply there when it is ready, and you then run it in Evals like any other eval.