When crowds act irrational there is usually a problem in communication. The crowd would still be able to solve problems better after they repair these communication channels. For instance, some rocket launches failed because information by engineers lower in the chain of command, did not work its way up to the decision makers. The group was too compartmentalized, but launching a rocket is necessarily a group effort. 9/11 could have been prevented, or the aftermath lessened, if communication between intelligence agencies was better. In a market crash we often see a single actor making a decision or prediction, and there is little to no reward for people down the chain to disagree with that prediction, or even adjust it (leading to insufficient variance in the predictions). Everyone is blindly chasing the experts, while in a good group setting there is no need to chase the experts.

> Have we ever looked at a tough scientific problem and came to the conclusion that the best path forward was to collect as many random people off the street and shove them in a room to solve it?

Has a scientist ever solved a tough problem growing up in isolation to other scientists? I consider "standing on the shoulders of giants" to be a form of group intelligence. But yes: We have done something similar at RAND corporation. The problem was: Forecast the impact of future technology on the military. The solution was to collect experts (not random people), put them in a room with an experiment leader, and gradually converge to the best forecast, using anonymous feedback every round. It's called the Delphi Technique and it is still in use.

Also, there is an experiment running right now, that takes random civilians, has them answer intelligence questions ("Will North-Korea launch a nuke within 30 days?") and gives weights to their answers, according to previous results. This way random civilians trickle up to the top, that individually beat a team of trained intelligence analysts, simply using their gut or Google. It's called the "the Good Judgment Project". Put ten of those civilians in a room and you have an intelligence unit that is not afraid to be wrong, does not have a reputation to uphold, and does not care about any group pressure, authorities or restrictive protocols that may hamper a group of real intelligence analysts.

> Also bootstrapping or model parameter selection techniques are already heavily used in AI

I believe the parent was talking about model ensembling/ model averaging, not ensembling techniques used by single models, like the boosting or bagging that random forests use. If you have a single attack as input crafted for a single model, then a voting ensemble of three models (lets say: random forests with Gini split, regularized greedy forests and extremely randomized trees) will not be foiled.