They do, but most people also live with other people. If you're immersed in VR, you're isolated from those you're with in real life. It's harder to keep immersed, if your g/f, wife, or kids hit you up every couple of mins.
So unless they also have headsets that are interacting with you, it's likely, that both you and they want to spend time with each other in real life, and when you go into VR, you're isolated from them.
