I think VR is for everyone, like any medium, it's just that you haven't found a game/app that makes you come back.
For me it's the multiplayer games that I like, more of my friends are getting VR headsets right now and I suspect that as more of them get into it the more we'll end up playing VR together.
Oh no, don't get me wrong, I'm totally mind blown by what I've seen. No Man's Sky and Skyrim VR most notably were just game changing compared to what I experienced in flat screens.
But yeah, VR requires my full and undivided attention, and this will just not happen :)
It's the same for me and gaming in general, I have very little patience except if I'm playing warzone or rocketleague with friends. It's just not the same :)
If you see VR as another medium like a laptop or a smart phone, then I believe they are for everyone.
People underestimate the amount of applications for VR. But Mark my word, in a few years your comment will have poorly aged as every household will have a VR headset the same way every household had a printer or a desktop PC 20 years ago.