If too many of them leave, the site becomes useless.
The people who answer (and upvote) questions like "What's your favourite programmer cartoon" do not think this through - they just see something funny and click the upvote button. People have argued "but this got upvoted so it must be good" since the site started, but it just isn't true.