I kinda get it if I move my mouse away as it shows more of the participant video. But if I am hovering over the mute button don't hide it!
It is very frustrating.
On GNOME I have set up a hotkey to mute my microphone (this is available but unbound by default). It is amazing as it works for all applications and doesn't matter what window is focused. The only downside is that you don't get the mute indicator.