It is, because you cannot use Signal without giving them your mobile phone number, and from that point onward they (and anyone they might be sharing data with) know the who/what/when, and more. My gut feeling, notwithstanding any apologist and their weak arguments, is that the design choice is exactly about the who/what/when because it's mandatory despite being entirely unnecessary from a technical perspective.