> you can't trust people's judgment about themselves and their skills.
Unless the candidate is fresh out of uni, any qualified candidate with experience should be able to provide at least several solid contacts at previous employers from which to obtain this information.
References from co-workers aren't really that valuable. Many people have no qualms about giving glowing reviews for friends, even if those friends don't really deserve glowing reviews.