Main memory access take an order of hundred cycles. D1 cache hit access usually take usually 3-4 cycles. Microarchitecture designers will take heroic efforts to even shave a single cycle here. Adding an overhead of even a couple of cycles would be a huge deal.
Having said that, AMD CPUs are the existence proof that you can be immune to meltdown with no significant overhead.
Spectre is a completely different issue though.
