Post-mortem debugging with coredumpy

Live-debugging is great, but what we often need is post-mortem debugging — which means debugging the program after if fails. Imagine a common scenario, when your tests failed in CI or your colleague’s machine, but won’t reproduce on your local setup. Won’t it be great if a dump can be generated for you to examine and investigate the point of failure?

coredumpy is the tool to do exactly that.

It dumps all the frames and variables so you can check what went wrong in VSCode debugger or with pdb.

That seems easy — just pickle the whole thing and load it right?

Well if you have ever played with pickle, you’ll know that it can’t be used in this kind of scenario in production.

The most fundamental issue is that, not all objects are pickleable. There’s no support for partial pickle, which means you can’t dump the data as long as a small piece is not pickleable — that’s a death sentence for its production usage.

It also has a few other issues — it does not deal with recursive data structure well, it is unsafe by design and it requires the same environment to load the data. It’s a great tool, just not for this case.

coredumpy choose the other way — consider all objects as pure Python objects, with some exceptions (basic types). You won’t be able to restore everything, but you can dump everything. The worst case scenario is that you can’t read the actual data of some special objects, but you can still load the others. If some types are critical to you, you can even write your own type support to serialize and deserialize it — yes coredumpy is customizable.

That’s my philosophy of a product — make it work, then make it better.

I also believe the best product is the one that is easiest to use, so it’s extremely easy to use it in your pipeline (or local setup).

With pytest, you can just do

pytest --enable-coredumpy --coredumpy-dir ./dumps

Or if you want to hook it on exceptions or unittest failures

import coredumpy
coredumpy.patch_except(directory='./dumps')
coredumpy.patch_unittest(directory='./dumps')

And of course you can just dump it whenever you like

coredumpy.dump(directory='./dumps')

Load the dump with VSCode extension, you’ll have a live-debugging like experience

Or if you are a pdb lover

coredumpy load your_dump

coredumpy is still being developed. It can be used now (I’m using it in CI of viztracer already), but if you have any issues/suggestions, feel free to raise it on github:

https://github.com/gaogaotiantian/coredumpy