Neurodiversity is probably aligned with that but I suspect a lot of people are just not self aware, never learned to communicate in a political environment.
A neurodiverse, even introverted kid who went to private school then Ivy League, and had 'a lot of money / no fears' growing up ... probably will have been socialized in the political sense.
Grow up in a small town or the burbs, people are 'judgy' about some stuff, but not anything complicated or nuanced.
I'm not in any camp, but really not until I was 30 did I notice how much stock people put into small signals. Like 'likes fishing' or 'football' etc..
Especially in such a competitive environment, people are I think looking for flaws.
Torvalds, James Gosling, Wozniack, Sergei Brin - those are nerd-nerds.
(I once saw Sergei Brin walking down the street in what looked like a back brace and mime outfit. Serious. WTF? Middle of the day.
Sundar is All Birds Nerd, but he's a business guy anyhow.
I feel a lot of YC culture errs on the side of All Birds Nerds.