Why you should try VizTracer to understand your python program

Have you ever thought about “why is my program doing this thing” when you are developing? That happens to me quite a lot when I’m working with Python, and becomes the reason I built VizTracer.

VizTracer is helpful in many aspects, debugging, logging or profiling. The fundamental idea behind VizTracer is to help you understand what your program is doing.

Unlike traditional debuggers, which normally pin you at a specific time and allows you to check on variables, VizTracer focuses on the call stack over time. Basically, VizTracer visualizes how functions calls each other while your program is running.

Recursive Fibonacci

This is a very simple example, the program is doing a recursive fibonacci number calculation:

def fib(n):
    if n < 2:
        return 1
    return fib(n-1) + fib(n-2)

VizTracer can show how the recursive function actually works.

VizTracer is not a toy, it can also deal with very large program.

Monte Carlo Tree Search

Above is the result of a MCTS(Monte Carlo Tree Search). You can zoom in to check the details of the program

Details of MCTS

It’s extremely easy to get this report. All you need to do is install VizTracer from pip:

pip install viztracer

And instead of running your program using Python, run it with viztracer:

viztracer your_program.py

VizTracer will setup the necessary stuff and execute your program, then generate an html file in that folder, which you can open with Chrome. No source code change.

Of course, sometimes call stack is not enough. You want more information of your program. You can also achieve that with VizTracer.

VizTracer can log all your function input arguments and return values. Simply do:

viztracer --log_function_args --log_return_value your_program.py

Then when you click your function entry in the report, you will find arguments on the bottom left corner:

fib(5) = 8 (this is older version, newer version has slightly different format)

Sometimes you will find the report has too many data which you are not interested in. For example, you only want to log your own code, not any library code. You can still do it without any source code changes:

# --run is added to resolve ambiguity
viztracer --include_files ./ --run your_program.py

If you are already familiar with the command line arguments and want something more powerful, VizTracer provides inline features, such as a counter that can log any numerical data and display it in a separate signal on the timeline.

counter = VizCounter(get_tracer())
counter.a = 10 # automatically logged
# blah blah
# ...
counter.a = 20 # automatically logged

I have a demo for logging “cost” value of gradient descent and it looks like this:

cost of gradient descent

You can clearly tell the cost is getting lower over time.

These are just some examples of what VizTracer can do with minimum effort. VizTracer has a lot of other features to explore and I’m adding even more into it.

Just like I said earlier, I’m not building a toy, so performance is also an important consideration.

The core part of VizTracer is built in pure C so the overhead is similar to cProfile, which I assume most of you are already familiar with. The worst case overhead is about 3x(with only function entry/exit). The average overhead is below 2x. You can achieve even lower with proper filters. And of course, if you want to log more stuff, there will be larger overhead.

It supports Linux/MacOS/Windows now and the test coverage is 99.5%+ (not a toy~).

Maybe it’s time to try a new tool to develop python :)

If you find a bug, or have feature requests, welcome to raise an issue.

https://github.com/gaogaotiantian/viztracer