In my culture it’s considered extremely casual — and therefore quite rude in social situations other than the most private and familiar — to talk with someone while wearing sunglasses. I can imagine the same thing would apply to AR devices too.

What you describe sounds like it could be a real problem, but one I’d blame on rudeness rather than Meta. We already live in a world where people order coffee while reading E! news on their phones.