Firstly, it's not "giant amounts of binaries" it's "a very small amount of binaries". A few GB is enough to cause significant problems.
Secondly, This _is_ an issue with git. If my project requires binary files, git should handle it. How should we handle logos in a mobile app, branding images on a website, audio files for background? That's before you get to the question of "how does a video game store the source version of a 100GB worth of compressed assets?"