The illusion of self can be observed directly. Perhaps it's even an inevitable conclusion of sufficiently intense introspection. I'd consider observing this to be a spiritual practice.
The metaphysics of realms of reincarnation, hungry ghosts, etc. is religious thought (and it was the dominant worldview when Siddhartha was born). These are not ideas that I can discover independently through introspection; if I believe them it's because someone told me to.
We might be talking past each other, though: if you're saying that the benefits of meditation are inherently inextricable from a Vedic worldview, I don't agree. But if you are just saying we haven't yet figured out which parts of the religion are actually necessary, then I agree. I personally learned meditation in a fairly Buddhist context, and naturally there are parts that resonate with me and parts that don't.
