undefined | Better HN

0 pointsgen2206y ago0 comments

This was a super fun read, thanks for sharing.

The mailing list piece is from 2012, and describes how git is very slow on a synthetic repo with millions of files and commits. Today, my current place of work has a monorepo that’s approaching the size described in this mailing list, but git seems to be holding up just fine. If you checkout a branch that’s far enough away from master it takes a minute, but add, rebase, commit, status and blame are all negligibly impacted speed-wise. The only issue we run into is rejected non-conflicting pushes to master during peak hours, with maybe several dozens of engineers trying to merge and push master simultaneously.

Does anybody have any insight into what’s changed in git internally since 2012 to support bigger repos?

0 comments

beagle36y ago

I don’t think there is one single change that made a huge difference. I follow the changelogs posted on the mailing list, and of the performance related changes, it’s often “we got 3-5% speed up on this benchmark on this fs without making things worse on others”.

Over 8 years and tens of those changes, it adds up to a significant performance improvement.

_0w8t6y ago

Git works nicely on Linux for Chromium with over 540K files spread across a few modules. On Mac and Windows it is kind of tolerable, but with git status taking 5 or more seconds, I started to use “git status directory” to get more instant feedback. And git blame can take more than a minute, so it is often better to look at the log and guess the changes from it.

yencabulator6y ago

Free RAM has a huge influence of how much of the filesystem tree is cached in kernel. This is visible from just `time find`. It could just be a case of developer workstations going from e.g. 4 GB to 16 GB.