Also, the IK-system would not be able to mimic the exact movement of your own arms, making them feel foreign and weird.
After some time, you forget you are even holding controllers (on the Rift, I'm not sure about those clunky Vive controllers, and I haven't tried the Index).
Vive owner here.
Depends on the game. In a game like PokerStars VR, yeah I often feel like I'm holding a controller. But that's not a game that aims to immerse you, as it's more of a social game. The real draw to that game is interaction with other players. VRChat is also in the same boat.
In any sort of shooter, not at all. Definitely still immersive and I don't feel like I'm holding a controller. This also applies to games where I'm holding things a lot, like Gorn. I feel like I'm holding inflatable toy weapons, which is fine because the weapons in the game flop around a bit like they're inflated.
Where in that scenario would you say you are confused about the position of your elbow?