Show HN: Time travel debugger for web development (opens in new tab)

If you can trigger a debugger from a keyword it's usually pretty easy to trigger it conditionally or on an exception in the code rather than through a GUI.

That works well for ruby and javascript and probably lots of others.

e.g. something like the equivalent of

  def foo
    bar.map do |b|
     b.do_something!
     debugger if b.state == :something_youre_interested_in
    end
  rescue => e
   debugger
  end

I'm really excited by Replay. I think it will be invaluable.

Err, conditional breakpoints are your friend.

Also the GP claims "nobody" uses breakpoints? Everyone in my team uses them all the time.

Suppose it all depends on your language/tool chain, in my experience breakpoints are very commonly used by developers in visual studio.

Which debugger do you use because I live in JetBrains tools and their debugger front-end support for conditional or on exception break points is phenomenal

For Python, but only if execution is deterministic, I made a small library that tries to address that issue: https://github.com/breuleux/breakword (it prints data alongside a deterministic list of words and you can set a breakpoint on a word of interest in a second run).

It's really a poor man's replay, though. That tool looks really slick, I'll definitely give the Python version a go if/when it comes!

Err, conditional breakpoints are your friend.

Also, "nobody" uses breakpoints? Everyone in my team uses them all the time.

I think print statements are actually useful in ways that typical debuggers are not meant to be; they make it easy to show changes over time, and they provide a tight feedback loop between observing the value of some data and performing interactions that update that data. For example, if you wanted to know how a coordinate calculation changed as you scrolled the page, print statements would be more useful than a debugger. I don't think this is exclusively why debuggers get less use, but I think that print statements aren't inherently a thing to optimize away from.

Couldn't agree more. Debugger support in modern codebases has become a huge after-thought which is such a shame.

It is an amazing way to discover how a codebase works. You pick a point of interest, and then you get the entire path from the beginning of the app's execution to that point as your stack trace, and every variable along the way too. Watches are great too for tracking a value changing over time.

Micro-services and Docker also took debugging many steps backwards - one advantage of a monolith is that you can easily step-through the entire execution, whereas if you have to cross process-boundaries it becomes a lot more complex to properly debug.

I'm working on a monorepo template at the moment where everything is debuggable with a single-click. This includes debugging native addons in C++ and Rust for Node.js projects. It's not easy - which is why people avoid debuggers so much.

I recently setup debugging in a Rust project for IntelliJ was the alternative was adding `dbg()!` statements which involved 10s recomplilation. The difficulty was implementing the right pretty-printers in lldb so you could see the value of various types because support is quite immature at the moment.

It's certainly a combination of these things. I use breakpoints all the time when I'm working with C# because I'm inside Visual Studio. It's super easy to work with the debugger there. With Source Link I can even step into other libraries of ours. Debugging C++ under VS is also not bad, and Python in PyCharm is a good experience.

But if I don't have VS or PyCharm available, I'll switch to printf debugging.

Though there are some cases where even with a good debugger I'll end up debugging by modifying the code. Sometimes it's necessary for performance reasons. Conditional breakpoints when debugging C# are extremely expensive so tossing one on a line that's executed many times may make the process far too slow. In that case it's better to compile in an if statement and then drop the breakpoint inside there. Other times the debugger is just limited in what information it can provide. Pointers to arrays in C++ are a common annoyance since the debugger has no length information.

My theory is that breakpoints are not useful because they let you go forward. But if you have an issue somewhere where a variable is not in the right state, it's because somewhere in the past was the issue. But you can't go back with a normal debugger.

Replay allows you to go back in time which is to me the biggest breakthrough. This actually makes them useful!

Maybe 2a. Executing code (like a print) in an auto-continuing breakpoint action makes the program itself pause; especially tiresome when you're looking at a timing or performance issue.

Just my anecdote: Personally I don't like using the one in Xcode (and maybe I'm missing something obvious) because I got so used to the debugger in JS land where I get access to a live REPL which functions just like the code I write. In Xcode, I'm stuck with some lldb prompt which I don't understand and definitely doesn't function like the one in JS tooling. I'm sure it could be more useful if I invested more time into learning it, but the barrier is there.

I’ve used good debuggers in the past, but the main downside to me is that the workflow improvement is relatively minimal compared to print debugging. The “live programming” aspect of Common Lisp and Clojure, as well as the way Cider implements tracing for Clojure _is_ a major improvement, but only because they let you be more precise in what needs to be re-run for print debugging.

I think it's often that the compiler/environment do not leave enough information for the debugger, typically by optimizing out local variable names and their values. By the time you figure out the obscure settings to be able to see the live values of variables and other state, you may have done a lot more surgery on your build system and slowed things down to a crawl compared to adding a few print statements.

You forgot one: the debugger is auto-updated with the rest of the browser and is changing UI all the time.

Are there any good books on using debuggers effectively?

You're right on the mark here. I saw rr used at Mozilla to diagnose and fix flaky tests that failed just often enough in CI to make life miserable, but were nigh-impossible to reproduce in a local development environment. Being able to take that to the next level and collaboratively investigate a bug using a recording that captures it is game-changing technology from the future.

Imagine a world where instead of ignoring, skipping, or marking "known failure" on all those flaky tests your CI hits (we all have them) you could capture recordings of them, and then actually investigate and fix them! That world is possible!

(Replay engineer) Google is currently the only way unfortunately. We wanted to support secure logins at launch, with SSO and Multi-factor Authentication, and focus most of our efforts on the core product. As a result we just went with Google for launch.

We don't currently have any plans to add additional authentication mechanisms but we've heard this feedback from a couple folks and we'll sit down to prioritize it after the excitement of launch has died down. Sorry about that!

(Replay engineer): we basically record all the inputs and outputs to a program. In the example of an HTTP request: when recording we'd record that a request was made, and the response. When replaying, rather than make the HTTP request, we return the response that was recorded.

If you're interested you can learn more about how Replay works here: https://medium.com/replay-io/how-replay-works-5c9c29580c58

That's the point: looks like a tool made to be the best of both worlds: You get access to the full debugger, at any point in time, instead of clicking "next" a hundred times and then not being able to go back if you missed something. Here you go forward or back in time, and you can visualize where you are. This thread sums-up much better than me why people use print debugging (not just because they don't want to learn a tool, or are dumb), and why Replay is the best of both worlds: https://twitter.com/geoffreylitt/status/1438152748449636360?...

I'm inclined to agree - I use breakpoints all the time when I'm debugging JS code and it is most definitely an invaluable tool.

It can be a bit cumbersome to setup and it can be a little buggy especially when you're working with transpiled code, but to say no one uses breakpoints is a bit disingenuous.

(Replay employee here): We have experimental support for some react-specific features using react-devtools since React is such a common framework, but the Replay is a general debugging tool and will work with other frameworks well too.

(Replay engineer): Yes it does!

(Replay engineer): Yes, replit has been working on adding similar functionality (https://blog.replit.com/debuggest).

In fact: we've been collaborating with them on it! :)

Yep! Tons on the site. Here's a Replay with an algolia search bug. Try adding a print statement in Hits.js!

https://app.replay.io/recording/cf2da81f-3a74-45ba-9241-f6d4...

Long story short, we forked the major browsers so that when you record we can capture the browser input (OS Library calls).

Brian walks through our approach here https://medium.com/replay-io/how-replay-works-5c9c29580c58

Wow! Thank you so much for this comment. I have been using IntelliJ for over 10 years and I never knew this feature existed, I just gave it a try and it's incredibly useful.

One thing I wish Java debuggers supported was the ability to move the instruction pointer to a different line, as has been possible in other debuggers for ages. Is it a JVM limitation maybe? I remember being able to drag the "current line" pointer forwards or backwards in languages like C, C++, and C# in maybe 2003. I wish I could do this with Java; dropping the whole frame is useful but this feature lets you do a lot more, like break out of a loop or skip a block of code you _just_ realized shouldn't execute.

Do you support it only for dynamic languages, where possible to monkey patch code?

All the debuggers mentioned above for the backend work only under Linux, because from what I understand, they use `ptrace` syscall, and on Mac have completely different format, and different capabilities.

Do you plan support Golang, especially on Mac, maybe with custom fork, or similar?

Thank you!

What, no! I'm a broke college student, I don't want to pay for this, at least not yet.

The video next to "Print statements from the future" on the webpage shows a bit of it.

You can click on the line-number next to a line of code to add a print at that location. If you hover over the line-number, it'll show you a count (exact and correct) of how many times that line of code hit during execution. Clicking on it adds a print log at that line with a default string.

At that point, the console log in the left should change immediately to include a bunch of new entries with "Loading.." text that resolves to the text of each print statement.

Clicking on the string allows you to replace it with an expression to be evaluated. The expression can include references to local scope variables, etc.

If you edit the expression, the console entries for the prints go back to "Loading.." and the new values are resolved.

The prints in the console are ordered in time sequence along with all the other events that can be shown there (regular console logs, mouse and keyboard and other events, etc.)

I asked about this (whether it could be done as a browser extension) and the answer I was given was that the browser was required because they do things like hook syscalls and other close-to-the-metal stuff

From: https://www.notion.so/How-Replay-works-cc65abf5eb11443586abb...

  So if you can’t capture API calls directly, what do you do? You drop down one level and record the browser system calls. This probably sounds like a terrible idea. We started off with a simple 3 line JS program with one API call and one clock and instead of just recording these two calls, we’re now recording a program with millions of lines of C++ and the complexity of an Operating System. Yep! It’s crazy, but works, and it’s pretty awesome.

  So the nice thing about system calls is there are not too many of them and they don’t change that often. This means that instead of recording an API call directly, we can record the network engine’s system calls to open socket and process packets from the server. And by recording at this level, it’s possible to replay the website exactly as it ran before, with the same performance characteristics and everything else. It’s a little bit like putting your browser into “the Matrix” and tricking it into believing everything is normal, when in fact it is just running in a simulation.

Session Replay records the DOM so it can be replayed like a video. Replay.io records the browser input so the browser session can be replayed again.

The biggest difference is that when you're viewing a replay, we're re-running the identical browser on our backend. This way you can debug the real thing.

I agree there's a lot of session replay style equivilents (and rrweb is great!), though what Replay is doing unfortunately cannot be done in production in the same way. In the ideal world this level of data could be extracted from production incidents without an individual running specialized software, but with the state of technology I'd say they're both aiming to solve different problems.

Thanks, edited a bit for clarity. What is a better way to say it?

Thanks for the explanation! Do you ever run into performance issues with replaying from the start on each backward step or is this not really in issue in practice? I imagine for most websites and short replays it's probably fine, but for something like a game with a physics engine it sounds like it would be too expensive and you'd need snapshots or something. I guess that's a super small percentage of the market though.

For question 3 on the ordering, I was imagining the following kind of scenario: one thread maybe calls a system library function to read a cursor position and another calls a system library function to write a cursor position. So even though they're separate functions, they interact with the same state. Do you require users to manually call to the recorder library to give the recorder runtime extra info in this kind of scenario? Sorry if this is a dumb question, I haven't really done any programming at this level.

As bhackett confirmed, you're right about recording at the system library call level. I wasn't sure if it was more of an analogy or only referred to a version of Replay targeting backend servers written in other languages like Go, especially since the author mentioned hooking into the JS runtime in https://medium.com/replay-io/effective-determinism-54cc91f56.... But it looks like I misunderstood, and their browser product is their generic record/replay library integrated into Firefox, rather than a reimplementation of the same concepts.

It's definitely a well-considered approach. I don't have to care about the framework that's been used or set up a custom proxy to capture and replay / simulate API actions, you're literally recording system-level calls and that would be extremely valuable for replaying those annoying edge-case race conditions that seem nigh-impossible to track down without significant effort at replicating exact steps and timings.

I mentioned in my other comment about Windows support, but even better if you could do something like browserstack, where I could just direct users to a URL where in the backend you guys are running the replay browser, but from their perspective they're just "using the website", that would be a killer feature. "Here, go to this URL and make the bug happen again, as soon as it happens click the little bug icon" - wouldn't have to convince an IT department to allow custom software on their COE, I could foot the bill and pass it on in my invoices so don't need to convince their accounting to approve licensing, etc, and you wouldn't need to compile OS-specific clients...

Anyway I digress, really cool stuff and thanks for expanding a bit on how it works, taking something so low-level as syscalls and wrapping it up in a user-friendly interface is no mean feat - good luck!

Cheers for the heads up on cypress, not sure how I've missed it but I've added to my reading list and I'll be doing a deep dive. I'm using redux on my current side project and loving being able to scrub through state changes.

My angle on this thread is that I don't have much control over the frontend frameworks and usually when I land in an enterprise integration job a lot of the broader architecture is already in place - longer term I can have some influence but generally debugging involves numerous stakeholders. A tool like replay seems useful because most testers / users are reporting issues from their direct experience using the frontend, and being able to record and scrub through their interactions would be a massive timesaver, in lieu of setting up custom testing frameworks etc.

I get what you're saying though, and I'll definitely check out cypress, just wanted to add some context.

That's a big selling point for me, props to the engineering team for accommodating this, I work mainly on integrations and the trickier bug reports are weird edge cases that involve a lot of getting specific descriptions of what happened and replicating it manually on a sandbox so as not to impact API calls, getting on the phone to talk through exactly what they're doing, asking for screenshots via email, clarifying conflicting understanding of in-house terminology etc. You know the story.

Being able to get them to record the exact process and scrub through it at my leisure without having to worry about hammering APIs would be a massive timesaver, replication is about 90% of the time taken to fix a bug, while the fixes themselves are usually trivial. Not having to worry about accidentally replaying a bugged-out API call is a huge plus.

In my case the major hurdle I can foresee is the majority of my enterprise-level clients use Windows and a brief look at the website says currently there's only Mac / Linux support for the replay browser. What's the timeline on Windows support?

The only other thing I can think may be an issue is that the recordings are cloud-based and a lot of the clients I deal with are finnicky about exfiltration and governance, and would be a lot more comfortable self-hosting where possible. Is that possible or on the roadmap?

This looks like a super useful tool to add to the toolbelt for sure, I would love to have all my clients using something like this to report issues. Nice work all!

I prefer email. Why do I need to sign up anyway?

Not with a 3rd-party provider.

Very interesting. I can change a breakpoint to not break, but log, instead. This should be quite interesting. Also the option to "go back in time" by dropping a frame from the callstack is very useful. This way, it's possible to go over some part of the execution again without having to restart the whole application.

Windbg-preview supports TTD too.

We've come a long way from the web replay days. We re-wrote the recorder to support additional runtimes like Node.js and Chrome. Replay is also cloud-based, so recordings are shareable and super fast.

We're really grateful for the support we had within Mozilla in the early days, but what we've come to learn is that projects like Replay really benefit from being able to be nimble and solely focused on a great experience which is difficult when you're a small feature in a larger product.

Also you can find our runtime forks and entire frontend on github. http://github.com/RecordReplay/

Happy to answer any additional questions.

Thanks for reporting I'll take a look at what went wrong there.

I just visited the recording and the upload seems to have completed - I can view the recording from your link and debug it. I assume you shared it publicly?

Understood, thanks

I'm using just a slightly outdated browser, which is probably the cause. I'm on Ungoogled Chromium 81 (by choice) which doesn't automatically update, and the ||= (boolean OR assignment) operator wasn't added until 85.

Your payload appears to compile down for older browsers (as is evident by the use of `var`, unless.. you actually write code using `var`) but it misses the ||= operator transformation it seems.

Thus, the browser throws a syntax error, which your app interprets as "new version" for some reason.

By the way, please don't disable right clicking. It doesn't actually solve any problems and only annoys users.

Hi Jason, This looks great. Recommend you add an email waitlist for windows version. My team only uses windows. So we cannot try it out till that is released.

I'm on Linux and I only get the Apple download button (on FF, and I have not messed with the browser ID string).

Ok thanks for clarifying that. Best of luck for your team.