Fragmentation means that many apps I want would no longer be available through a vendor that I trust. No, that trust isn't blind or absolute. Yes, I recognise that Apple's review process is patchy at best—but most developers also fear getting on the wrong side of them and THAT alone has been pretty damn effective at keeping most apps under control.
I'm not convinced this is as big a problem as you think it is. If I chose iOS over Android specifically for privacy reasons I'd be interested in knowing what companies avoid the app store to bypass their privacy restrictions. Do you really want to use a product from a company that would do that as soon as they have the chance or would you rather label them as untrustworthy and find an alternative?