> Sure, the work is chaotic. But why would the interview process need to be chaotic?
Because the interview is meant to measure how well someone would perform on the work?
> You want to get the best signals of ability that you can, and one thing you can do to help that is to make sure that you’ve given an assignment that will be evaluated consistently on your end and understood uniformly by the participants
Well sure, all else being equal, but if the cost of consistency is an assignment that doesn't reflect the actual work environment then that may outweigh the benefit.