When the player performs a skip, it waits for stream packages whose timestamps match the new position after the skip. It’s the client who requests a different segment of the video, and waits until it receives the respective segment, as identified by the embedded timestamps. Skipping isn’t a purely server-side operation in that sense, the client side has to cooperate. The server has no control over which timestamps the client wants to play.

The only other alternative is to make the video a live stream of indefinite length where the user can’t skip forward beyond the farthest point they already played.