Like FizzBuzz I think the take home worked well as another data point. It easily filtered out people who couldn't follow basic instructions, people who were clueless about Java even though their resume said different, and people who simply didn't care. The problem also gave us something to talk about when the candidate came in for the in person interview.
Overall the problem was dead simple for anyone we would actually want to hire.