Fair point. I'd say the frustration comes more from the double-talk of "look how inclusive our workplace culture is: everyone eats together in the cafeteria!" The company admitting "we don't have any norms around lunch; everyone is on their own" would not be an objectively better experience, but at least would be honest.

Another option would be to have more official team, group, or project-based lunches, rather than a free-for-all, where specific members of a team privately invite other specific members of a team.

I don't mean to turn this into a rant, just adding a data point :)