Technically:
* Doesn't need hardware performance counters - runs on more CPUs and on cloud systems (where performance counters are often blocked).
* Can attach and detach at any time - means you get to record just a subset of program execution that's interesting.
* You can our ship recording tech with your application and control it by API, so you can grab crash recordings on customer systems.
* Supports programs that share memory with non-recorded processes.
* Supports direct device access (e.g. DPDK).
* Accelerated debugging features - searching with recordings using parallel processing, accelerated conditional breakpoints a few thousand times faster than native GDB.
* We provide a stable, patched fork of GDB that we're occasionally told is more stable than the default.
For many people's use cases none of these really matter - they should use RR if they're not already.
But if you need any of these things then Undo can give you time travel debugging. In practice, it's usually big software organisations that we deal with because they have development pain and the extreme requirements we can match.
That’s actually pretty neat.
Undo has cool features like Live Recording that we don't have in rr. They don't need access to the hardware PMU which is a big advantage in some situations. They can handle accesses to shared memory in cases where rr can't. https://undo.io/resources/undo-vs-rr/ is a good resource.
It may not be commercial quality but its open source and free :)
[I built rr.soft]
[1]: https://undo.io/resources/undo-performance-benchmarks/ - "Undo serializes their execution"
Before it was released publicly I believe Microsoft had been using it internally to share recordings on bug reports against massive pieces of software like Office. So it's a serious piece of tech.
https://sourceware.org/gdb/current/onlinedocs/gdb.html/Proce...
But it's limited. It's really cool that it's integrated by default but it doesn't scale to big applications / workloads.
RR and Undo both use GDB as a user interface, though, so any skills you have there will carry over.
All these Oh wait. I missed it...debugging sessions. and these What exactly changed over there? are answerable.
The category namer of time-travel debugging, TimeMachine, (hence time-travel debugging in contrast to other attempted names such as reversible, bidirectional, record-replay, etc.) was available in 2003 and supports/supported the ARM7 [2]. Note, that is not ARMv7 architecture, that is the ARM7 chip [3] in use from 1993-2001.
From what I know, the ARM7 was one of the first ARM designs implementing the Embedded Trace Macrocell (ETM) which could output the instruction and data trace data used to support trace probe-based time travel debugging.
[1] https://jakob.engbloms.se/archives/1564
I've thought I bit about how you might support time travel on bare metal embedded - but actually there are hardware-assisted solutions (Lauterbach's Trace32 was one we came across) there sometimes.
Pricing & Licensing
A UDB floating license costs $7,900 per year.
Undo is mostly used by companies whose world is complex enough that rr doesn't work for them, and they understand how powerful time travel debugging is.
There has now been a LOT of engineering invested by a lot of very smart people into Undo, so it does also have a lot of polish and nice features.
But honestly, if rr is working for you, that's great. I'm just glad you're not doing printf debugging the whole time :)
I was in talks with them recently because I kept running into limitations with rr. The main advantages for my use case were that undo doesn't have the same dependency on hardware timers, which means the ARM support is much better, you can run it in a VM (e.g. a cloud machine) and you can do replays on different systems.
This, of course, ignores employee benefits and overhead which usually amount to ~100% extra costs over direct pay. So that is now ~1.1% and ~2.1%, respectively.
And that ignores the fact that you need to pay people less than they produce to be profitable which probably drops us down to ~0.5% and ~1.0%, respectively.
[1] https://www.levels.fyi/companies/google/salaries/software-en...
edit: Incorrectly linked to product designer instead of software engineer levels.
