Tracing Python (opens in new tab)

Python has a built-in API for tracing: https://docs.python.org/3/library/sys.html#sys.settrace

Here's an example of how to use it (obviously a minimal case, you can customise your handler do do whatever you need):

    def mytrace(frame, event, arg):
        if event == "call":
            print("call", frame.f_code.co_name, frame.f_locals)
        elif event == "return":
            print("return", frame.f_code.co_name, arg)
        return mytrace

    import sys
    sys.settrace(mytrace)

For example, given the following code:

    def three(z):
        return 3

    def two(y):
        return three("BBB") - 1

    def one():
        return two("AAA") - 1

    one()

It will output:

    call one {}
    call two {'y': 'AAA'}
    call three {'z': 'BBB'}
    return three 3
    return two 2
    return one 1

What's the modern practical use case for tracing in python? Environments without an IDE/remote debugger?

I can't recall the last time I needed to enable tracing, in the last couple of decades, besides bash scripts.

There are many different tools, but when I want to quickly understand what the code was doing I have been reaching for pysnooper: https://pypi.org/project/PySnooper/

Has a pretty nice and importantly colored output by default, which I find much easier to follow. In many cases snooping one specific function has done the job. It has an easy to use decorator to decorate any function without having to otherwise intercept the main python invocation or set something up at the start. Good for libraries etc.

pysnooper has mostly been enough for me but the "snoop" mentioned in the article seems to be a superset and more. Will have to give it a go next time.

Outside of this if I want to understand performance or more roughly what code paths are executing (and not a per-statement trace) then austin and py-spy are both amazing.. convert to flamescope and drop into flamescope.app.

A more traditional debugger for Python.

https://github.com/inducer/pudb

I'll throw in my own too: https://github.com/kunalb/panopticon

Fantastic article and something I've been looking for better tools for. None of them seem to have my ideal workflow though -- providing an external manifest of what and how to trace that can be used against an existing program without changing the code. Something like Otel's auto instrumentation.

Fyi, the table of contents gets cut off (its box is smaller than its contents) at zoom levels of 90% or larger for me. Here's a screenshot at 100% zoom https://imgur.com/a/0t9cKs5

Great article!