It would be a lot more work, but one could check the validity of ratings based on how other users rate things compared to this user.

Say there are 4 games. Most users who rate all 4 rate them similar, except for the 4th game that always gets really low. So 5,5,5,1 is a normal expected rating, but 1,1,1,5 isn't. So 5,5,4,4 from a high rater or 2,2,2,2 from a low rater would be given more weight than a 1,1,1,5. Other things can be added such as weighting a user's ratings a low impact if they have too few scores to determine ratings from.

This reminds me of the problem of determining the answer key to a multiple choice test given only the answers of the test takers.

This does actually happen, possibly for several other reasons. Bought ratings often come accompanied by other ratings, whether bought or not, just to attempt to cloak the bot. This even happens in systems where there aren't any countermeasures being taken to remove bots yet - I was vaguely involved in spam detection in one and saw all sorts of cloaking behavior even before we'd turned on any sort of mitigation.

Yea, that's the main reason why I'm not sure it could be made to work.