I wonder if there is any chance that Apple will change how this works - it seems like this should be doomsday for a large swath of apps.
(married to Smokey and partner on all these apps!)
There is no current safe place to save local data that should not be backed up to iCloud.
The app: webalbumsapp.com
From what I see, your users have 3 choices: They can use your app and use up their iCloud allotment, they can use your app and not upgrade, or they can upgrade and find a mapping app that is more efficient in its use of space.
One can probably easily imagine an interface for showing the user how much data a particular app is using and allow them to nuke the temporary stuff. It might even look beautiful. It might even be fun to use. But it's going to introduce a lot of hand-wringing and micro-managing and lots of mental overhead that Apple really, really wants to avoid.
On the other hand, if users and developers in a community are free to view, modify and share the source code of the operating system and programs they are using, then that creates a dynamic where software cannot remain defective by design, something which cannot be said about Apple's hardware or software.
If we were talking about free (as in speech) software, users would have control of their data, and their choice of how to manage it; not some distant programmers working for a corporation's bottom line.
Is it just me, or has the free software community gotten so disciplined and precise with their talking points that it would make Karl Rove blush?
It comes down to this: different people want different things. Some people want Kramer's vision of building your own pizza, both for fun and to get exactly the pizza you want. Most people would rather have pizza driven to their house after a 1 minute phone call, and if they lose some freedom over the exact topping placement and the kind of crust used, they're quite fine with that.
Yeah, this was a really boneheaded platform decision, and I'll be very frustrated if Apple doesn't fix it. But you can't seriously tell me that open-source software doesn't sometimes make blunders or suffer from flawed usability. And the idea that I can theoretically crack open the source to my app (the Venn intersection of those with the time, ability, and willingness to do so are a speck, by the way) doesn't help me after the cache is already cleared while I'm sitting on the plane.
I don't care whether something is defective by design, or defective by lack of design: I care that it's defective, period.
The day that technology vendors start engaging in things like free speech suppression or non-consensual privacy invasion is the day I'm willing to think of computing as being a political issue. What we have here is nothing more than tech which you think sucks, and in this instance, you're very right. If it sucks enough, people will stop buying it, and/or the vendor will make it suck it less. But for god's sake, get off the the soapbox and go back to making things; everybody has exactly as much freedom as they want.
tl;dr: FOSS needs to get better at delivering pizzas instead of bitching about Domino's.
Example Gnome3, which is free software, even a officially sanctioned GNU project. Overnight, a few key people decide to abandon the traditional desktop metaphor, against the will of a majority of their users and to the detriment of the whole linux desktop. The users hated it and are still hating it, but nobody has the ressources to modify it. Nobody has even the ressources to keep Gnome2 alive, let alone fork Gnome3.
Yes, in theory users could make only those changes they want, but only in theory. In practice, if a significantly influential player says that Gnome2 is gone, Gnome2 is gone. Even if iOS was free software, if Apple decided to make a change, they would be still big enough to force a majority of users to accept their way.
Circumventing design decicions is hard. Learning the specifics of particular design is hard practically in the same way. Hacking in an intentionally prohibitive environment is not much different from hacking in an overtly permissive environment — there's a pile of poorly undocumented design decisions which you have to grock before you get anywhere in this system.
So the actual difference it political. I won't argue that preventing little Joe to write the coolest Doom clone ever by requiring all software to be cryptographically signed before running it on a particular game console is somehow more evil than forcing him to spend years learning the ropes and working late hours in QA.
But I believe that imposing certain constrains on people willing to build on your platform is a good thing, because the resulting uniformity gives the platform more leverage. Whethere the constraints are technical or political /in origin/ is irrelevant.
Every non-apple developer and user is required to bend to
the will of the programmers who develop the proprietary
iOS.
Every non-developer user is required to bend to the will
of the programmers who develop the OS.
Developers are not users. I don't particularly want my mobile phone to respect the "freedom of its developers". I do want to be sure that no app I install at a whim hogs the disk down, installs spyware or f*ks the phone when I'm in need to make an important call. If I wanted "developer freedom" I'd choose OpenMoko. Strangely, millions of us did not.
> Every non-apple developer and user is required to bend to the will of the programmers who develop the proprietary iOS.
He's only "required" so if he chooses to use iOS. And if he chooses to use iOS, it's because the platform attracted him, and one thing that attracts a lot of people to it it's the "it just works" and "you don't have to manage it".
> Any negative choices or limitations imposed by Apple Inc. are virtually uncircumventable.
Yeah. Except if you jailbreak.
> On the other hand, if users and developers in a community are free to view, modify and share the source code of the operating system and programs they are using, then that creates a dynamic where software cannot remain defective by design, something which cannot be said about Apple's hardware or software.
Yes. Then you get the mighty OpenMoko. Not defective by design. Yay!
> If we were talking about free (as in speech) software, users would have control of their data, and their choice of how to manage it; not some distant programmers working for a corporation's bottom line.
Yes, for a brief time in college I'd quote RMS too. You'll get over it.
edit: I'm shocked this thread is so long and no body mentioned this. It's been on the apple developer forums for months as a solution.
https://twitter.com/#!/marcoarment/status/124587437799374848
In fact, if you use UIManagedDocument, it creates a .nosync directory in the document bundle to prevent it from syncing in the framework itself.
Has anyone used this successfully?
The articles themselves, yes, send them to the cache. If the user needs to reclaim the storage used up by the articles, let the OS delete them. Then, when the user needs to read the article again, it will take some time to download. But don't get into an "all articles gone" situation. Just my 2 cents.
I'd say in this case, the content is user-generated and should be backed up.
But this does indeed defeat the point of having it offline, which is a significant part of IP's purpose.
With respect to point #1, and in the context of what Instapaper is for, the user is intending to read an article offline, and the local copy of the article was generated by the user's intent (and can, therefore, be considered user-generated even though the user is not the author of the article.)
With respect to point #2, the articles fail the "can be downloaded again" test in light of the app's purpose of making the articles available to the user when the network is not available. When the network is unavailable, the articles cannot be downloaded again. Edit: JeffDClark makes another excellent point here that the article may have originally been behind a paywall and therefore cannot be guaranteed to be re-downloadable. The same is true if the author removed the original article from their webserver.
Ergo, put them in the Documents folder. Whether this would satisfy the app reviewer is another question, but it's worth a shot to carefully explain to them how your app is not violating the letter of the law.
That is the title of Apple's main iCloud page at http://www.apple.com/icloud/.
iCloud is seamlessly integrated into your apps, so you can access your content on all your devices.
This is Apple's definition for the iCloud service. It doesn't matter what the data is, it's your data and Apple is promising to sync it between your devices, to preserve your experience.
In the case of Instapaper, the solution is obvious: Put the files in Documents. That is Instapaper's content and part of the experience that users want synced between devices.
If Apple penalizes developers and undermines the promise it is making to users because it decides to be miserly about bandwidth, then it has to admit it launched iCloud before it was ready.
If you go to the bottom of http://www.apple.com/icloud/features/documents.html you can see the web interface you have to use.
I know this is a tangent to the OP but I was very annoyed to discover this last night, I feel like Apple was pretty misleading here. There are clearly technical difficulties that still need to be surmounted.
That's the key right there. The size of the Instapaper database is determined by the user. It will only fill up a user's iCloud space if they decide to save a lot of documents. Plus, they always have the option to buy additional space!
Aside from making sure the documents are reasonably compressed, the size of the Instapaper database should not be Marco's concern.
1. Only documents and other data that is user-generated, or that cannot otherwise be recreated by your application, should be stored in the <Application_Home>/Documents directory and will be automatically backed up by iCloud.
2. Data that can be downloaded again or regenerated should be stored in the <Application_Home>/Library/Caches directory. Examples of files you should put in the Caches directory include database cache files and downloadable content, such as that used by magazine, newspaper, and map applications.
I don't know if this is something enforced in the approval process, but it seems like Apple's intention is for redownloadable content to all be stored in temporary, wipeable locations.
If people don't want the benefit of having their Instapaper content transfers across their devices automatically (and I do think this is a good feature) then they are free to disable iCloud backup for Instapaper in Settings.
To avoid this issue and enjoy the benefits of iOS 5, I'm going to have to clear out a bunch of apps, music, and photos and ensure that I always have a sufficient amount of "buffer" space so the cleaning is never triggered. I cannot be alone in this, and the fact that Apple is making this kind of thinking necessary for end users is kind of ridiculous. I really can't see this behavior lasting for very long, and I'm sure Apple will address it soon; this is the antithesis to the traditional iPhone experience. The only scenario where I could see this being purposeful is if Apple is really trying to hurt offline apps to increase data usage and appease carriers (maybe for pissing them off with iMessage?).
That way, the default behavior is that "download something = want to keep it on device", but users can do the other one too if they want. I don't think the option is particularly useful, but it might make the app reviewer happy?
This is literally exactly how I would implement the choice if I were Marco.
[Edit] Upon further examination of the new Settings/iCloud in iOS 5, what will need to happen is that the user should be able to disable iCloud syncing for individual 3rd party apps. That's the most consistent solution for the current interface. [/Edit]
1. Temp: No backup, cleared regularly
2. Cache: No backup, cleared when space is tight
3. Local: Local backup only, never cleared
4. Documents: Local/cloud backup, never cleared
5. Cloud: Cloud backup, cleared when space is tight
The problem seems to be that #3 doesn't exist. Yet you'd think it would be a common requirement for stuff like in-app purchases of large and essential content packs, for example, turn-by-turn navigation maps.
I'd hate to be on holiday and have a 10 megabyte podcast download automatically trigger the erasure of 1000 megabytes of navigation data.
Or to put it another way, if it's important enough to retain when storage space is low, it's important enough to retain for when you have to recover from a backup.
I would guess the idea is to help enable these 500 MB per issue magazine downloads. You download a new issue, you nuke some old issue, no one cares. As long as that wasn't an issue you cared about.
The more storage space taken up by local files being backed up to iCloud, the more people will end up going over their 5GB capacity, and will have to upgrade to the higher capacity plans to cover it. I'm not sure I actually believe this right now, but if they refuse to fix the problem, then I might...
It seems that the argument that only the list of metadata is user-generated can apply to any type of media (music, movies, etc...). Technically speaking all of the music on my phone could be downloaded on demand. Of course this requires an always connected, fat, and cheap network connection. Which is pretty much the opposite of what most folks have.
This also breaks the user's expectations. I was annoyed/surprised when I upgraded to iOS 5 and Instapaper had to re-download everything.
As far as I know, all scraped articles are stored in Instapaper's server and downloaded from there, you don't get them directly from the source page.
The argument against that is that you're now syncing that article with iCloud in addition to Instapaper.
But I wonder whether the correct answer is instead to eliminate Instapaper's sync feature, and just let iCloud do it. Once you have system-level cloud sync, don't you want to let Apple do the work? Sync is hard, and it isn't really the core value of Instapaper.
Edit: I was wondering about iCloud only for iOS 5/MacOS 10.7.2 devices with iCloud accounts. But that story does fall apart for people with mixed devices. So never mind.
Maybe Apple will release iCloud for Snow Leopard, but as it is now, instapaper works all the way down to the terrible version 2 iPhone I have had to resort to using in order to not sign into a new contract waiting on the release of the 4s.
Instapaper would hurt too many current users who are just fine with how things work without the cloud.
http://developer.apple.com/library/ios/#DOCUMENTATION/FileMa...
That said, I think we're in the clear with respect to iCloud backups:
"Devices with an active iCloud account have their app data backed up to iCloud at appropriate times. And for devices that are plugged into a computer, iTunes performs an incremental backup of the app’s data files."
It's ambiguous. But "app data" sounds like documents, whereas "app's data files" sounds like big data files in /Library to me.
The relevant part of the documentation is here: http://developer.apple.com/library/mac/#documentation/FileMa...
I wonder if OS X, in line with its trend to be more like iOS, is going to start automatically clearing the ~/Library/Caches directory as well.
Perhaps Apple could have made cache cleaning opt-in on a per-app basis until iOS 6, though.
Many people would prefer the old method, where it would say you were out of space, and you would have to go free up some space.
My Nintendo Wii uses a "blocks" concept for this. It works fine, Apple should just copy it.
I guess Marco just prefers venting after the fact.
However, he's being asked by multiple people right now for the radar# and he's ignoring the questions. He just said he's known it for two weeks, which might have been too late for Apple to fix it anyway.
There is a category of data that is aimed at offline use. Streaming apps like Spotify, that let you download playlists for offline use. GPS apps that download hundreds of MB of map data. You get the idea.
On one hand, this data is a form of cache. The data is always available elsewhere (on the content provider servers) and it can be restored if necessary in a worst case scenario. But the key word here is "offline". This is the kind of data that, by definition depends on being around if the user is offline and therefore cannot be easily restored on demand, when the user needs it.
Obviously, having all of this stuff backed up to iCloud and using up GBs of people's capacity is not feasible or even logical. So this kind of data does not belong anywhere that iCloud will back up. But it must be stored somewhere that is safe from being purged.
Yes, a users GPS maps can be restored eventually but that doesn't help them when they are stranded in the middle of nowhere with nothing but a weak GPRS signal and all of their maps gone.
Apple have made an almighty cockup in overlooking the "offline data" use case.
In Marco's case, I'd agree that the articles represent user data that should be stored somewhere like Application Support, which will be backed by iCloud but I think that's probably fine in this case.
* the lack of persistent, non backable offline storage * the addition of a auto-cleaning procedure
The auto-cleanup without a storage alternative is a blunder. iOS 5 decides for the user what should be deleted to free up space. An alternative would be to prompt what application data you would like to remove instead of a blind decision. Otherwise, that's the kind of programming that will one day lead robots to cleanup the human race! ;)
Of course, having permanent local storage that's not backed up would be a more user-friendly solution. However, it would probably lead people to run out of space when developers begin abusing it like some did with temp or cache storage.
It seems like a trend at Apple these days. With Lion, the OS now decides when an application should quit when unused.
The ball is in Apple's court.
Solution: I disabled backups of rdio data.
[1] http://37signals.com/svn/posts/347-youre-not-on-a-fucking-pl...
I just don't buy the lame justification about not always being connected. If you don't want to be connected, turn off your damn phone. Don't insist that everyone else shouldn't be able to enjoy their time the way they want to because you associate your mobile device with stress.
2. It goes against the spirit of Apple's policy here, which is not to backup to iCloud what ca be re-downloaded (see 1)
3. The user might have lots of content stored and again due to 1, those don't need to be backed up to iCloud. And really Instapaper actually has a small version of the problem. What is critical is applications that has huge amounts of data like offline maps and wikipedia. Those can be re-downloaded from their servers yet most definitely should not be backed up to iCloud due to time and cost.
Would that work?
This is an interesting twist especially with the 16GB devices which tend to actually be above 50%.
In the same way Apple doesn't want every developer operating their own independent payment system, they also don't want developers operating their own cloud storage services. If Apple holds the data they can guarantee its security. They likely see these problems as temporary, until developers and customers learn to adjust to the fact that iOS data is controlled by the iCloud service.
your app absolutely needs tons and tons of data to function? doesn't seem like your day. it's their device, their cloud, their decision. Apple doesn't give a shit about your day; they're going cloud. they may be wrong, but i'd guess they're going to have to find that out for themselves.
i'd assume they acknowledge this may kill some apps. i don't think they ever promised anyone a business; on the contrary, they seem to remind developers that they are there at their good grace all the time. as someone that built Facebook apps since '07, trust me, i know what this is like. start coding and start calling. best of luck.
I'm confused. If the goal is for documents to be available in the near future in offline form, why not keep documents in /Documents until the user has read them (or some sane amount of time has passed), and then move them to /Caches or some temporary storage?
I've never used Instapaper, so perhaps documents are only stored until read anyway?
Of course, one could argue that the cleaning behaviour fundamentally defeats the purpose of a lot of apps, as already covered above (Instapaper, Offline Maps, etc.).
My iMac is packed up for building work, so I can't upgrade my iPad and check this out for myself. Sorry.
Not exactly.
Why dont you just update your app?