I don't think it's that big a deal however because, in practice, libraries that use inheritance as an external API are almost an anti-pattern at this point. I forget who said it but someone (Josh Bloch? Scott Meyer?) said at one point "inheritance is an implementation detail". Or maybe it was "inheritance breaks abstraction"? Something like that anyway.
I 100% agree with this. At that point it doesn't really matter if your methods are final or not. Ideally you make your leaf classes final and move on with your life. But designing for inheritance is probably not going to end well regardless other than some notable utility classes (eg AbstractMap).
So instead of just setting up and executing the call, you have to traipse through memory a bunch of times, affecting caches and making things slower.
But the thing is, Java's compiler works at runtime, so it can either inline the function (basically dump the code into the call site instead of invoking) or devirtualize it (if the same object is called over and over, the JIT can just remember the last lookup and add a guard if the target changes).
In say C++, making everything virtual would be crazy because the compiler can only safely inline in certain situations and can rarely devirtualize a call.