What did the app need the contacts for? I'd say I side with apple on that (I can see how it could be abused to shut down competition though). There really would need to be a good reason to have the contacts. (I don't want to debate the threshold, just interested in a "benign" example of needing contacts)
Calling, texting or emailing said contacts from inside the app. Having this data was for the exclusive benefit of the end user, and the permission was optional and did not block use of the app.