You're probably correct. I've been very lucky to have mostly been around smart people, and so it's likely that the average middling person I come across is, well, just that. Middling status-climbers are still dangerous, though, and still work to ruin averages.
That said, Haskell actually has a lot of great resources available, and I think it is learnable by just about anyone these days. It usually only takes one good book for a language to become available to average people, and Haskell has a great volume of beginner-friendly books. It can be debated as to whether this is a good thing or not, but I think it's always nice to see new pathways into programming become available, even if I'm not necessarily in love with a language.