- Technically deep enough to reveal useful information
- Fair to the applicant (no free labor/conflict of interest issues)
- Time efficient for both the interviewer and the applicant (doesn't take a week of life to interview for a single position)
- Maybe give indications on other important things? (work habits, social interaction, etc)
My off-the-cuff thought was to ask for them to work on a smallish issue on an open source project they haven't contributed to before. After the work has been done, they'd come in to be interviewed about what they did and how it worked out. If this was a standard process, they'd be able to use the same work for multiple interviews, so job hunting wouldn't be such a huge burden. It also gives them a chance to showcase a few skills that I've found important in my own career (navigating a new codebase, integrating and testing a bugfix/enhancement, interacting with team members). It would also have the nice side effect of providing some motivated help for open source projects.
That was just a quick stab at the issue to use as an example, I'm sure it has a number of fatal flaws. The question is, could there be a solution that would actually work?