True, but as someone who has been part of that ecosystem: your standard Java developer doesn't rebuild everything. They don't want, need or have to do it. The Maven repo is immutable and very reliable. So everybody builds on top of the binaries.
Of course, there's this implicit system of trust, which might be misguided, but there's many billion dollar companies built on top of it.