EDIT: To clarify what I meant, HMDs typically use techniques like ATW/ASW/etc (link posted by Rebelgecko) to do just-in-time correction of the rendered image. The end-to-end motion-to-photon latency of the entire pipeline grows but the part that generates "sick" is very short.
In the parent's case, 20ms latency from movement to visible motion is part of the pipeline that:
- reads input
- evaluates solution
- returns solution to your screen
All kinds of things add to this latency: polling frequency of your input device, bus speed of the device, how fast you can update the world, how fast you can render the update, how quickly that updated image can be sent to the screen, how quickly the screen is able to turn this into visible light. etc.
