"If the app store can truly compete with an open marketplace"
As it turns out there is an open market for iOS apps and an open alternative to iOS and Apple's walled garden is competitive with both.
"why won't apple let me install arbitrary software on my iDevice?"
Aren't the reasons obvious? Simplicity, profit, security?
You might as well be asking why your Macy's card doesn't work at Sears or why you can't make a copy of your building key at Sears or why.. hey just what does Sears have against freedom anyway?
First, what is this open market for iOS apps you mentioned? I haven't heard of it.
Of course Apple locks down the app store on their iDevices for profit. I think we're in agreement on that. If they didn't limit software delivery in that way, their app store would have to compete with other software delivery channels including the open web.
What I'm trying to express is my surprise that developers in the desktop world would opt in to a market like this. I had assumed that iDevice developers only used the app store because they had no choice.
I doubt that Apple is solely concerned with profit and even if they were I guess I don't see that as the end of the world.
If Apple is solely concerned with profiting from apps why do they subsidize free ones and credit card fees? {If dev share is actually calculated post cc fees I'll plead ignorance and move on to the other part of the argument.}
99 cent app less 69 cents to developer less 40-70 cents to credit card company less bandwidth fees = loss.
(I've given a range on cc fees because they are shared if you purchase multiple apps within a few days, that mitigates the loss but ultimately someone motivated only by profit would be taking easily avoided losses all over the place in either scenario.)
Isn't it well known that Apple's direct profit on the App Store is a rounding error compared to their other business?
Isn't it then more likely then that their motivation is not primarily to directly profit from it?
While I do think Apple is interested in profit here, I very deliberately listed it as secondary to simplicity which is really at the heart of everything Apple does well. There's a book coming out soon about just this:
http://kensegall.com/blog/2012/02/and-now-a-different-kind-o...
The premise I'm building on is that Apple really profits by making great products that just work and make people happy. Having a super simple way to get apps is part of that.
For 98%+ of users 1 app store >> 2 app stores >> 10 app stores. One App Store just works. It also stops most malware without any action by the user. That's simplicity again.
"What I'm trying to express is my surprise that developers in the desktop world would opt in to a market like this"
I guess again I'll fall back on my Sears comparison. I don't know if Sears still sells software but they did for decades -- Wal-Mart still does for sure though and there's no way Wal-Mart is any more dev friendly then Apple. Way harder to get your product on the shelves at, harder to keep your product there, takes at least as large a cut, etc.
What Wal-Mart does have is tens of millions of customers though. Apple too. So of course devs are going to opt into those markets.