Farm it out to many people in return for some tiny (but not monetary) reward.
OKCupid used to have a feature for "choose my best image". Users submit two images, then rate other people's images. One of your images is presented alongside one other image from some other user, and someone else has to pick which one they like most. OKC then used their stats to give some analysis to it.
It was sort of great, except there were usually too few people to get any kind of sensible results.
It's easy to see that this could be a useful approach.