That's true, mostly because of the JVM's garbage collection. SaberVM also hopes to be a good way to run untrusted code, like the JVM, but it can run much lower level, higher performance code safely, using arenas and the theory powering Vale. AOT compiled SaberVM bytecode can have optimization passes that remove many of the checks, like Vale has now, and that can still be done on the client's computer in a safety-preserving way.
The JVM also comes from an era of OOP being a very pervasive norm, which helped Java's popularity a lot. My impression is that the JVM is seen by many as an annoying bottleneck and massive dependency that's the cost of using Java, a language they enjoy.