My perspective is that I used to think like you growing up as a Creationist, and I can't see a major difference between the arguments OP is making and the arguments that were made to me by members of my church. I used to be highly dismissive of group consensus, I used to say that if you couldn't explain your point in a way that convinced me in specific, that I didn't care how many people agreed with you.
I now think that consensus is an important metric that should be at least considered, particularly in areas where I am not an expert.
Of course, we've seen areas where the broader scientific community was wrong. Before race science became a fringe argument used by fringe segregationists, it was a generally accepted conclusion. But again, underconfidence is just as dangerous as overconfidence. You can't pick one, you have to reject both.
> you reached your conclusions based on the output of the academic system (presumably).
At the end of the day I have to base my conclusions on something, and I think basing them on scientific output is a better place to start than basing them on ideology. Even from the much more subjective metric of "do I trust the communities who argue for this", climate-science doubters and gender-science doubters don't come out looking well.
During the early rise of neural networks, where consensus was that this was extremely promising for AGI, I could find neutral, intelligent people that I respected who disagreed with that conclusion. That made me feel much more confident rejecting the general consensus myself. On the subject of gender-science, I can't find the same number of quality defectors. To me, the people saying that race/gender are heavily tied to IQ look a lot like Creationists, and I ain't getting pulled into that trap again.
