I don't think that's right. Client side certificates can be used with any domain. There isn't even an X.509 attribute to represent such a restriction. No major TLS or certificate store implementation I'm aware of provides any out-of-band way to restrict client cert domains either, not even the PKCS11/Cryptoki hardware interface.

If you have client certs installed and ready for use, especially with automatic selection, a rogue but otherwise "trusted" server can request your certificate by its issuer DN and, even though you may not directly provide any other information, any details about your identity present on the certificate can then be seen by that server.

Even so, thanks to the underlying security model of TLS, giving your cert to a rogue server still doesn't directly open up any confused deputy or MITM risks though, as far as I know, which is more relevant to the comparison with Yubikeys. Certificates, even client certificates, are meant to be "public", and the mere possession of one proves nothing; no certificate should be trusted until the party presenting it can prove it has the associated private key.

Corroborating SO answer: https://serverfault.com/a/1086000

I mean a separate physical device, like, well, a Yubikey, that can't be automated away due to some vulnerability or UI spoofing. A browser keeps your client-side certificates. A browser is a hardened, but also an incredibly complex piece of software. Chances that an exploit would let coax it into activating a particular client-side certificate without your noticing are pretty slim (hopefully), but for a hardware key which is simpler and even more hardened these chances are lower still.