Fun fact: Rust actually had a proposal at one point to execute drops entirely statically, with no runtime flags on the stack. It was decided against because people thought it would be too confusing as it would be hard to tell when the destructor would run, but for purely memory related destructors it would probably be acceptable.
I do see what you're trying to get at, I think, but it's also worth noting that the use of stuff like arenas and vectors to absorb the cost of the repeated reallocations goes a long way here towards making deallocation times predictable in practice (if not in theory). It is certainly the case that you can mostly reduce the deallocation overhead to ~ zero for any particular part of your Rust program without that much effort, unless you are writing an interpreter for a different language that expects GC semantics (at least, that's been my experience).