That makes a lot of sense, I guess it's either that or the adapter would have to somehow modify the EDID display capabilities to get the HDMI source to output PCM audio (or whatever the splitter accepts) only.
I wonder if it doesn't already need to do that for non-audio-capable HDMI sinks. (My first HDMI monitor didn't have speakers or even a headphone jack, and as far as I recall, the source computer could tell this somehow and wouldn't offer to send sound to the monitor.)