ah, the "sufficiently smart compiler" argument. There is no such compiler, FYI, and there may not ever be. Not for every situation that developers are creating in Java today.
it's a matter of hitting your CPU cache as often as possible. Neither javac nor the runtime has any freaking clue how to do that, but you, as a developer, do. When you write Java, you can't do a lot to control how that JVM arranges data or how it fetches it from RAM, partially because of OOP, partially because of autoboxing/unboxing, so you wind up missing the cache a lot more unless you create a bunch of primitive types and use those, and that that point, why are you writing Java?
I'm not saying Java is bad. I'm saying that it is not the fastest language you can use. You seem to be arguing that it is, or that it could be. It cannot.
> average performance for a large codebase will be much better than something like C/C++ given the same amount of effort.
absolutely not. Maybe if you're fully skilled in Java and new to C or C++, but if you are equally skilled in both, you would never choose Java for performance alone.