There are approximately 93 million adults (civilian noninstitutional population) that don't work[0]. I'm sure it's not all because of the employers themselves, but surely a lot don't want to deal with the politics and culture imposed on them by managerial talent [maybe some study on this portion of the population's reason for not working would be useful].
0: https://www.bls.gov/news.release/empsit.t01.htm (Men, 20 years and over->Not in labor force + Women, 20 years and over->Not in labor force)