> programmers have no idea what makes a good programmer
I disagree. I think we do know what we want, but there is no cheap/easy litmus test for this. If time, money, and opportunity were free, the way to find out if someone is a good programmer is to work with them for a long time.
Okay, now find a cheap way to do that while both sides of the interview have to juggle full time jobs and other interviews.