Pricing
Evals
What each model costs, what a whole eval costs, and how far a run can go before it stops by itself.
What you pay for
An eval costs the sum of three things:
- Model tokens: what each model reads and writes, at its price below. Taken from your credits as the run goes. With your own models, your service bills you for these instead.
- Machine time: each run's machine, by the hour, while it runs.
- Storage: only if you keep the machines after they pause. See Keep the bill small.
Ask this again on a trace is one extra model call each time, at that model's price.
Model prices
Prices are in US dollars per million tokens. Cached input is text the model already read earlier in the same run, charged at a lower price.
| Model | Input | Cached input | Output | Typical run |
|---|---|---|---|---|
| DeepSeek V4 Pro | $0.3447 | $0.0287 | $0.6895 | $0.019 |
| DeepSeek V4.1 Flash | $0.36 | $0.0072 | $1.44 | $0.022 |
| DeepSeek V4 Flash | $0.009 | $0.009 | $1.536 | $0.0051 |
| Kimi K3 | $1.308 | $0.864 | $16.80 | $0.116 |
| Kimi K2.7 Code | $0.8054 | $0.216 | $4.02 | $0.052 |
| GLM 5.3 | $0.12 | $0.0576 | $5.04 | $0.021 |
| GLM 5.3 Flash | $0.18 | $0.036 | $0.60 | $0.011 |
| MiMo V2.6 Pro | $0.522 | $0.0043 | $1.044 | $0.029 |
| LongCat 2.0 | $0.36 | $0.0072 | $1.44 | $0.022 |
Typical run is a run that reads 50,000 tokens and writes 3,000, which is about what a small bug fix takes. Real runs are often cheaper, because much of what a model reads is cached. A bigger task reads more and costs more.
A real eval, priced
One bug-fix task, three models, one run each, on Small machines. All three passed the check.
| Model | Time | Tokens in / out | Model cost |
|---|---|---|---|
| Kimi K3 | 1m 07s | 43.2k / 2,036 | $0.084 |
| GLM 5.3 | 1m 13s | 52.0k / 4,135 | $0.024 |
| DeepSeek V4 Pro | 30.6s | 46.1k / 1,872 | $0.0049 |
| Machine time while the models worked | $0.0010 | ||
| Total, with clean-up right after | $0.115 | ||
To estimate your own eval, add up one typical run per model per run, then add a little for the machines. Five models with two runs each is ten runs.
Machine prices
| Machine | CPU, memory, disk | While running, per hour | Paused, per month |
|---|---|---|---|
| Small | 1 vCPU · 4 GB · 20 GB | $0.0219 | $1.00 |
| Medium | 2 vCPU · 8 GB · 40 GB | $0.0281 | $2.00 |
| Large | 4 vCPU · 16 GB · 50 GB | $0.0472 | $2.50 |
A paused machine is free for the first 3 days. After that its disk is charged at $0.05 per GB per month until you delete it.
Keep the bill small
A run's machine does not stop when the run ends. It keeps running until 30 minutes pass with nothing happening, then pauses and keeps its files. For the eval above, left alone, that adds about $0.033 of machine time, then $3.00 a month of storage after the first 3 days.
To avoid that, once you have what you need:
- press Save & clean up in the desktop app, or tick Delete machines after I confirm the download on the website; or
- delete the eval.
Traces stay readable after clean-up. Deleting the eval removes them too. See Results and traces.
Limits
| Runs in one eval | 10 |
| Models in one eval | 5 |
| Time for one run | 30 minutes |
| Model calls in one run (BoltzLabs models) | 400 |
| Model use in one run (BoltzLabs models) | $5 |
| Time for the check | 5 minutes |
| Tasks in one set | 50 |
| Tasks × models × runs when running a set | 10 |
| Folder uploaded from the website | 48 MB and 10,000 files |
The per-run limits mean a model stuck in a loop costs a known amount: at most $5 of model use for one run of a BoltzLabs model. When a run reaches a limit, it stops and the check still runs on what it did.
Plans and credits
- Each run uses one machine slot, so your plan decides how many runs can go at once. A free account has one slot and Small machines only.
- In the desktop app, evals need an active monthly plan.
- An eval with BoltzLabs models cannot start when your credits are used up, and a run that is going stops getting answers when they run out.
See Usage and limits for plans, credits and machine slots.