Python tracer

1. Install

pip install git+https://github.com/boltzlabs/python-tracer.git

It needs Python 3.9 or newer and nothing else.

2. Record your model calls

Wrap the client you already use. That is the only change.

import boltztracer as bt
from openai import OpenAI

client = bt.wrap(OpenAI())

client.chat.completions.create(
    model="my-model",
    messages=[{"role": "user", "content": "2 + 2?"}],
)

It works with the OpenAI and Anthropic clients, and with any endpoint that speaks the same way.

3. Name the run

Say what the task is and which model is doing it. Run the same task with another model and the two line up side by side.

with bt.trace(task="fix-login", model="my-model"):
    run_my_agent(client)

4. Record your tools

Put @bt.tool on a function and each call to it becomes a step of its own.

@bt.tool
def run_tests(path):
    ...

5. Look at what happened

python -m boltztracer

agent fix-login      4.2s
  llm chat my-model  1.9s  1.3k in / 40 out
  tool run_tests     2.1s  FAILED AssertionError: expected 3 items, got 4

A failed step says why it failed, so you can see where a run went wrong without reading all of it.

No code changes at all

Put one line at the top of your program. Every OpenAI and Anthropic client it creates is recorded.

import boltztracer.auto

Good to know

• Traces are files in .boltz/traces, next to where your program ran. Nothing is sent anywhere.
• API keys and tokens that show up in a prompt or an answer are replaced before saving.
• A run that crashes or is killed still leaves its trace, with the step it was in.
• The tracer is for Python today.

Costs, sub-agents and OpenTelemetry are covered in the README on GitHub.