Again, the issue of compile times is more complex than "just do this one thing and it's fixed."
After you first cargo build, you have all of the dependencies pre-compiled. Every single "cargo build" after that's compile time would be completely unaffected by pre-built dependencies. You could try sccache; builds are still slow. Most people care about those secondary builds, not about the initial build, so telling them that this solves their problem is simply not true. It's not downplaying it, as I said, the first builds are also important. Unfortunately, it's just not that simple.
If solving Rust builds were as easy as this, we'd have just done it long ago. We care tremendously about improving compile times. It is a constant request on surveys. We put a lot of engineering work into this.