Correct - just like an evildoer who had your yubikey could generate and save a bunch of yubikey key strings, they could also generate and save a bunch of time-based codes for times in the future by changing the host clock.
You can use a bidirectional challenge-response between the yubikey and a trusted server - that's what U2F does.
But honestly, if an attacker has both your password and physical possession of your 2fa token, it's already game over.
