Programmers think of profilers when their program is slow, and debuggers when their program does unexpected things. However, there’s a nice combination of both tools - a tracer.
A tracer taps on certain events and records them on the timeline. With proper visualization, you can learn what’s your program doing at specific timestamps, just like watching a movie.
Different tracers tap on different events. For example,
ftrace on Linux kernel is most commonly used as a function tracer, while
strace traces system calls.
Today I want to talk about a Python tracer - VizTracer.
ftrace , VizTracer mostly taps…
Say you have a class like:
def __init__(self, x, y):
self.x = x
self.y = y
When you try to print an instance of it, you get something like
<__main__.Position object at 0x7f3289cabac0>
which is not helpful in most cases.
To solve it, normally we have to either print
__dict__ attribute, or write a
__str__ method of the class. However, writing
__str__ function for every single class is too much work for quick debugging, and
__dict__ is not flexible enough, not to mention it can’t deal with nested objects.
Profiling is a critical step when we try to optimize our code. There are a lot of profiling tools in the market. However, most of them share a common shortcoming — They provide too much information.
Don’t get me wrong, I’m not saying information is a bad thing. Sometimes we do need details to figure out why our code is slow, but for larger production level project, we are often more interested at “transaction level” profiling.
So, what is “transaction level” profiling? We should discuss “function level” profiling first, which is a more familiar concept for programmers. Take Python’s built…
You might be familiar with the chrome trace profiling tool if you are a front-end engineer and often work with Chrome. In fact, you can open chrome://tracing and record your browser’s behavior. The result will look like the image below:
It is a great tool for both performance analysis and debugging, because it shows every single thing happened in all the threads in chronological order. You can zoom out the get a general idea of the whole picture, or zoom in to check the details of a single function.
Have you ever thought about “why is my program doing this thing” when you are developing? That happens to me quite a lot when I’m working with Python, and becomes the reason I built VizTracer.
VizTracer is helpful in many aspects, debugging, logging or profiling. The fundamental idea behind VizTracer is to help you understand what your program is doing.
Unlike traditional debuggers, which normally pin you at a specific time and allows you to check on variables, VizTracer focuses on the call stack over time. Basically, VizTracer visualizes how functions calls each other while your program is running.
Debugging is an inevitable and often painful process when any programmer developing their programs. However, proper tools could make this process easier and help you to reach your destination faster and with less pain.
In this article, we will talk about a lot of python debugging tools that are still being maintained and could help you with your debugging process.
print() is still the number one choice for a lot of python programmers. I won’t spend too much time here because any python programmer should know how to use
print() to debug.
print() has many advantages, it’s built-in, it’s insanely…
I’ve shared the package VizTracer that I’ve been developing recently. I presented it as a profiler. However, it was more than a profiler. There are more aspects of it.
So today, I’ll take another angle and talk about it as a logger.
First of all I want to introduce a little bit about VizTracer to those who have never heard of it. It’s a python tool that can record data while your python program is running, and generate an HTML report of your execution like this:
The most important data VizTracer records is function entry/exit(or, FEE). It records every single…
My boss once told us, never optimize without profiling. That is very true in almost all cases. Without proper profiling, you may spend hours in optimizing and end up not improving your performance at all.
So, the first thing to optimize your code, is to find a good profiler.
There are a lot of options in the market, even when we only talk about python.
cProfiler, which is probably the most common profiler python programmers are using, is a built-in package that provides deterministic profiling. …