If you have fifty bright and highly competent people, I'm skeptical that adding fifty idiots is going to make much of a difference. Most idiots will accept meritocratic authority if they can be convinced that their needs are taken into account (which they should). Some will obstruct, but probably not enough to significantly derail anything, and the good-faith idiots will bring information and perspectives that wouldn't be considered if they weren't there, so they aren't exactly useless.

In fact, I would argue that idiots in elected bodies are a lot more likely to do damage than random idiots, because they are more likely to be narcissists, and being elected boosts their sense of self-worth. And of course, the most damage is often caused by the most intelligent of them, because the main problem is acting in bad faith, not a lack of wits.