Ask HN: Are there any good Diff tools for Jupyter Notebooks?

edit: Googling reveals nbdime, has this been looked into? - https://nbdime.readthedocs.io/en/latest/

Not OP but I can imagine easily the need for what he's asking.

You'll find a lot of algorithms for data and image processing saved as notebooks these days offered to you. Let's say you make some changes from the provided code and after a handful of changes something is not working right. You might want to diff from where you are back to a working version in hopes that differences that emerge might clue you into where to look for where the problem might be.

As an aside, I want to say Jupyter notebooks (moreso jupyterlab) is sort of a disruptive change to our coding workflows. We've had interpreters for a long time sure, but creating interactive graphs on-the-fly is a godsend, insights come to you in such a workflow that wouldn't otherwise. I hope this catches on, I actually want my shell terminal to become more Jupyter-like. Also, fun fact: did you know you could do real-time collaboration on Juypter notebooks? https://jupyterlab.readthedocs.io/en/stable/user/rtc.html

Not OP but restart and clear output can be quite compute intensive if you're working with big datasets or training ML models. There are many ways to mitigate this like saving weights and only redo the inference but it's not always worth it when you're iterating through models and parameters or doing exploratory data analysis. Most of the time you want to just keep results/outputs of previous run and improve from there

This with https://github.com/untitled-ai/jupyter_ascending + your Editor to have a supercharged notebook workflow.

I wish for a simple option in VS Code: On close of a Jupyter Notebook clear its output. Or something that separate the display of the output from the saved file (Still `ipnyb` file). See [1].

[1]: https://github.com/microsoft/vscode-jupyter/issues/9514

Woah this is exactly what I'm looking for! Thanks

I used this and can recommend it, since it also shows you the outputs of different versions.

But as another commenter said, when I got to the point of needing to diff my notebooks, I realized that I could move some of the code into separate python files.

If you're a business analyst, one use case is if you need to process some data e.g. every quarter, but the data changes a bit every time so you need to update the approach slightly (e.g. data structure changes, new mapping rules). With nbdiff it's easy to keep track of changes while having some helpful visualizations in the same file.

I use nbdime daily for notebooks under version control thanks to the following configs in my ~/.gitconfig (global git config).

   [diff "jupyternotebook"]
     command = git-nbdiffdriver diff --ignore-details
   [difftool "nbdime"]
     cmd = git-nbdifftool diff --ignore-details \"$LOCAL\" \"$REMOTE\" \"$BASE\"

I'm not sure if this is a standard setup or if I copy-pasted from some blog post, but overall it's working great.

There are some issues with it, like (1) will unnecessarily mark graphics as changed (e.g. re-generated figures from the same code), and (2) the diffs become less meaningful if large chunks of cells were moved, but overall it works great.

If it supported a `--color-words` option then it would be super helpful for seeing only which words have changes, instead of whole lines changed (very good for long paragraphs of Markdown text).

yep, can confirm, it basically filters out any notebook output from version control (while keeping it intact in the notebook file itself). This works seamlessly with diffing, committing, staging, etc.