What it is
We write questions whose answers are in your documents, such as manuals, contracts, reports or spreadsheets, and put the documents in the run’s machine. The model reads them and answers; each answer is compared with the expected one.
Document Q&A
Questions about your own documents, with answers checked against them.
Coming soon. This kind of eval can't be run or requested yet. Today BoltzLabs runs codebase evals: each task is a repository, a job to do in it and a test that checks the result, run on its own machine.