My limited understanding says a debugger needs: a list of symbols (.pdb files on windows, can't remember what they are on linux), understanding of syscalls and a few other similar things. I thought they don't care too much what generated the binaries they are debugging (obviously as long as it's native code).
Doesn't rr work with other languages like rust, zig, odin, nim, and similar ones? Obviously, I wouldn't expect it to work for python, js, c# and other languages with managed memory.
*Edit
Found this, https://zig.news/david_vanderson/using-rr-to-quickly-debug-m...
https://github.com/python/devguide/issues/1283
https://morepypy.blogspot.com/2016/07/reverse-debugging-for-...
https://github.com/mesalock-linux/mesapy/blob/mesapy2.7/READ...
Things that don't work are drivers that update mapped addresses directly. An example of this is CUDA in order to replay one would need to model the driver interactions (and that's even before you get to UVM)
Another great thing is that RR records the process tree and so you can easily look at different processes spawned by your executable.
(Unlike rr it's not open source though - sorry! We have lots of programmes working on it full time and they insist on getting paid every month :)
Tho, it feels wrong to expect a tool designed for native binaries to work well with python in this context. And that's ok. It feels lucky when it works as much as it does.
[0] https://robert.ocallahan.org/2020/12/rr-remix-efficient-repl...
I'm not familiar with Rust, but I'm almost sure it has a good C interoperability. If a certain piece of software is working well, what is the benefit of rewriting it in Rust?
This sort of property is nice to have in huge codebases where you really start losing confidence in shipping changes that don't subtly break things. But of course a huge codebase is hard to rewrite in general...
The "working" C program has a high risk of undiscovered bugs relating to concurrency and memory safety. Rust lets you rule out a large swathe of them by construction. Rust's type system is also far more expressive, which in many cases enables cleaner domain modelling.
One benefit is it's easier to hide malicious code thanks to Rust's complicated syntax.
Presumablly the idea here is to support Rust replay debugging, not just rewrite a C/C++ targetting replay debugger in Rust
I assume rr provides more features and flexibility. Anyway I want to mention that GDB itself can already reverse debug for some time now.
> runs out of resources
Still, thank you to share your first-hand experience. Did you ask the GDB Dev team for any feedback on the slow performance?
It worked, it helped me track down the bug, but it was painfully slow, I had to do things to limit the size of the input to make it possible to use at all (and thankfully was luckily able to still repro the problem after doing so).
i have been able to use gdb's replay functionality usefully because i had an input file which crashed the program within a fraction of a second after startup. this meant that i could navigate backward from "this variable is wrong" to "how did this variable get set to that wrong value?" in only several minutes of waiting on the computer
gdb reverse debugging was introduced in 2009 [2].
You can see a fairly comprehensive history of time travel debugging here [3].
Not to say the built-in gdb reverse debugging was any good. It had (has?) like 1,000,000% overhead which is basically unusable. At least some implementations in the history that were introduced earlier only had ~1,000% overhead or less in general. Yes, a literal 1,000x overhead difference.
[1] https://robert.ocallahan.org/2014/03/introducing-rr.html
Despite that, it would be very, very, very cool if some languages built rr directly into their tooling. Obviously you can always "just" use rr/gdb, but imagine if rr invocations were as easy to set up and do as pdb is in Python!
There is Undodb which works on Mac and maybe with multithreading (not sure about that), but unfortunately it costs about $50k.
https://www.forrestthewoods.com/blog/windbg-time-travelling-...
10% is 100x cheaper than WinDbg and cheap enough to leave on all the time in production. That is a game-changer.
If you’re gonna throw around numbers like this you need to cite an actual tool not “if I remember correctly there exists a unicorn”.
rr record /tmp/Debug/bin/llvm-mc a.s && rr replay -d cgdb
I've have success story with some bugs only reproducible with LTO. Without rr it would be a significant challenge.
It would be nice if Linux kernel could be debugged with rr. Does anyone have success with kernel under rr+qemu ? :)
> [...] just click on the incorrect value. With full program history Pernosco can immediately explain where this value came from. The value is tracked backwards through events such as memcpys or moves in and out of registers until we reach a point where the value "originated" from, and each step in this process is displayed in the "Dataflow" panel that opens automatically. There is no need to read or understand the code and think about what might have happened, we can simply ask the debugger what did happen.
https://news.ycombinator.com/item?id=31617600 (June 2022)
Super useful, especially considering I know barely anything about x86-64.