No one really fully knows WTF is going with iCloud - documentation is basically nonexistent, support unavailable, implementation broken in trivial ways, and error messages inscrutable. I was at an Mac/iOS conference recently where the Core Data iCloud talk morphed from a standard talk into a "here's the mic, anybody have any idea what's going on with this thing" affair. Helpful, but indicative of the state of things.
IMO, from having studied this for many, many hours over many moons, is that what Apple is trying to do is fundamentally hard, if not entirely unfeasible. They're trying to replace a smart server with one that's dumb as a doorstop. More specifically, they're trying to emulate a CRUD web service with a file sync engine.
Conflict resolution is left up to individual clients, since the server doesn't do any "thinking". Likewise, there is no canonical, authoritative state of the store, since the server doesn't "think", only the clients do. Apple was hoping that by shoving a bunch of diffs of your database onto the server, that clients can reliably reconstruct a sane database by playing them back - except that multiple clients are updating the diffs simultaneously and there is no server-side conflict resolution.
Oh, and if stuff fails, there are no regular snapshot states to fall back to, because iCloud is a file store, not a database engine. Your whole store is now corrupt. Enjoy.
Oh, and there's no way to nuke the database and start over - there are metadata files that are undocumented or that we don't even have sandbox access to, that interfere with completely destroying the database.
At this point you might think "Fine! I'll go to my Settings -> iCloud -> Manage Storage menu and delete everything associated with that app and start it off on a clean slate!"
And then you find out that, even today, in iOS 6.1.3 land, there is a bug from iOS 5.0 where if you delete an app's iCloud bucket it can never be created by that app, on that account, ever again. I wish I were exaggerating. In perpetuity. Presumably some state gets fucked and now this app is forever more unable to store anything on iCloud.
To make even the most trivial experiment on the iCloud API involves creating a brand new app.
iCloud (in so far as it relates to Core Data and document storage) isn't just buggy, isn't just poorly implemented - it's non-functional.
As a user though I have absolutely no idea what is going in with Apple syncing. My wife has an iphone / mac syncing explosion that I simply can't fix. I just don't have any idea where the data is or why everything is duplicated in her iphoto collection. Or what's going to happen in when you remove something from one place.
Most of all I cannot for the life of me explain to my wife why it is that when she lost her phone and restored a previous backup to a new one it DELETED ALL HER RECENT PHOTOS from off her mac without warning. Gone, forever. I watched it happen before my very eyes and it's made me want to never buy another apple product again.
Let me tell you it's not exactly happy land for us consumers either. Apple's "cloud" is a mess. And don't even get me going about the "new" and "improved" iTunes. Frankly I pity developers, cause Apple's mess is giving them a bad name.
I can't think of 1 icloud enabled app (on Windows or Mac) that I've used that worked well. The apps themselves are fine, it's the cloud integration that fails. I mean just look at Apple's own forums.
After 10 years as a Mac user, I'm actually considering moving back to Windows. I'm just over it. It's like every new OS (or app) something goes missing. I'm tired of spending hours searching for replacements for features Apple has removed. And I'm tired of being force-fed services and apps (that Apple's PR states is "gonna change my life") still in beta.
Ok ok //end rant
Apple has a long story of half-finished features they try to force onto developers and users. And if that doesn't work the technology gets killed off.
The only downside of not using shiny but questionable new features is that your app won't get featured in the App Store.
This is a pretty huge downside. The App Store is a huge mess - there is effectively no web frontend for it, it's tied deeply into a desktop application, and it's obtuse. This makes marketing nigh impossible and results in Apple having an enormous, and disproportionate, ability to pick winners and losers.
Being featured is a Big Deal(tm).
Of course, if this isn't fixed soon...
The fact that anyone, anywhere (including Apple), actually expected Core Data to work as a distributed object store is totally beyond my comprehension.
> I have no experience with iCloud, but I'm pleased with Core Data for managing a persistent object graph. It automatically handles version migrations, undo, and faulting, and some interfaces can be populating with almost no code via bindings. I'm happy with it. Still, I'll be very interested to watch the Core Data presentations at WWDC.
1) Core Data objects are not NSObject objects. You can't treat them remotely like normal objects: https://developer.apple.com/library/mac/documentation/Cocoa/...
2) Core Data requires making your model mutable, and spreading dependencies on Core Data's abnormal objects throughout your entire code base.
3) Since the abnormal objects are spread throughout your code, and they're tied to non-thread-safe NSManagedObjectContexts, concurrency becomes rather difficult. You have to deal with multiple managed object contexts becoming out-of-sync across threads. Most people just give up and perform expensive operations like -save on the main thread.
4) Core Data merges the concepts of your on-disk model with your in-memory model, despite the fact that the two problem domains have very different requirements in terms of API design, data longevity, relational design, etc. This leads to code that is neither a clean in-memory model, nor a clean on-disk model.
5) Since Core Data objects are abnormal objects, one must produce a considerable amount of boilerplate just to define new model objects. What time you save upfront using a GUI to design your original model, you spend 10-fold in maintaining the code to support it, and in being handicapped by the contraints Core Data places on your code.
6) NSPredicate is awful. It's a brain-dead query language that often make it impossible to cleanly normalize your data store. Aggregate operations are expensive and often require O(n) cost, the query language is poorly documented and poorly specified.
Core Data is a very poorly fit abstraction, but it probably works well enough for simple use-cases. Once you move into concurrency, complex/large models, aggregate operations/projections across your data, or have to deal with it failing, it falls over.
After using it myself for a large project and struggling with all of these issues (and more), and watching another team do the same on their own project, I'd never make use of it again. SQL (via sqlite) is a far better abstraction for managing on-disk data, especially when doing so in a highly concurrent application where one can rely on SQLite's transaction support.
Having dug through iCloud's underlying structure, I can tell you what they're doing is very close to what Microsoft's Sync Framework does - right down to the naming of some of the columns in the underlying database.
In fact, if I didn't know better, I would think it was a peer implementation of the Sync Framework (which considering the iCloud server is hosted on Microsoft Azure which has native support for Sync Framework built into hosted SQL Server, isn't too far fetched).
The difference is that Microsoft's Sync Framework seems to work.
"Distributed Databases are Hard" has been a mantra for many decades, probably getting close to a half century. We have known, basically forever, that this is hard on a fundamental level. (Google "Two Generals" problem.)
> Apple was hoping that by shoving a bunch of diffs of your database onto the server, that clients can reliably reconstruct a sane database by playing them back
This is called a variety of things. One of the terms is "transactional replication." It can get very very sticky when there is more than one party originating write transactions, though.
I just change the topic to a different backing store when the client wants "cloud persistance" and says "iCloud" without understanding the pain they're asking for.
I agree, completely non-functional
Not that I'm blaming the technology specifically, but my impression is that Apple is being naive.
Of course you can build with Java, but if you knew better and have the resources Apple has, you would be aiming somewhere else. Google can do this because the infrastructure is there.
Not in the case of Apple
And as this shows, the concept itself of how the service works is broken. This is smelling of 'walled garden fundamentalism' + 'technical ineptitude'
I know it's fashionable to hate on Java, for a lot of good reasons, but I think it's naive to dismiss the power of the platform itself.
We may all have (completely valid) complaints about the outdated and verbose syntax of the language, but the actual JVM itself is fantastic. There are more scalable, fault-tolerant, high performance Java systems out there than the rest of the implementations on all other platforms, combined.
Much of the iOS community still creates software by the seat-of-your-pants model (i.e., write code, boot it up, does it look like it works? ship it). Engineering rigor is still a concept relatively foreign to iOS, and it shows. Apple has little-to-no built in support for anything that smells like TDD.
When you're talking to your server-side peers who have good test coverage, good regression testing, and more generally just sound engineering processes, you can't help but get bit jealous.
And just for the record I have a few flagship apps in the app store with millions of active users
Source please. because your toolset is from yesterday and their's is from tomorrow
This is why I don't believe your first point without sources to verify. Eclipse+Android SDK is hardly from tomorrow. Windows 8 possibly. What else does that leave? Nada.Oprah Magazine (1M+ subscribers): https://itunes.apple.com/us/app/o-oprah-magazine-archive-dec...
Esquire Magazine (1M+ subscribers): https://itunes.apple.com/us/app/esquire-magazine/id433020707...
Those are just two that I have built. I have an enterprise platform that I architected from the ground up that companies like pfizer use for it's whole workforce. I have ones written for Disney, Hearst, GE, MaryKay, Hallmark, Sesame Street, etc, etc. The one I'm building now is ready to scale to ~5M users right out of the gate and I believe we will succeed.
Now put your apps up and let me pick through them and tell you why you're wrong because I just don't believe your work is credible.
Having started working on Android recently, if Android is tomorrow, the future sure is dystopian.
That's not to say that the iOS testing tools are great, but I've never had a particularly pleasant time testing GUI code on any platform.
And Apple's WWDC session about acceptance tests using Instruments UI Automation specifically handles UIAlertView.
I think the problem is that iOS apps are getting dumber and dumber while their UI bling gets more complicated/asynchronous. The value of tests is too low for The One Widely Accepted Testing Framework to emerge.
And what would that "toolset of tomorrow" be? Surely not Android's joke offerings, or Windows 8 same old...
It "just works" (if it actually worked).
With Dropbox, either you are storing everything for all of your users (and paying for it), or you integrate into the user's Dropbox account - which would necessitate a fancy ol' login/authentication system. Far from seamless, and relies on your users subscribing to something that is popular but far from ubiquitous.
So... either pay through the nose for all your users' storage, or force them to jump through some circus hoops on their own. Neither are great.
For apps that get a fair bit of off-line data creation this is a nice feature.
There's no way for a third party library to do this (at least not through iOS 6.1.3 SDK).
I haven't seen many complaints about iCloud's file syncing features, which are what Dropbox is the direct competitor to.
1) Google Drive - took my a long time before I even tried it, but it works about as flawless as dropbox for my needs. I actually like it a lot as I use Google Docs heavily, so it makes it easier to use those tools.
2) SkyDrive - offers the most base storage than the other. Has nice support for MS Office products.
3) AeroFS[1] - While it's still in beta, it looks promising (it has limited (android) to non-existant (iphone) support). [1] https://aerofs.com/
Those 3 things are broken as well. Lost all my data twice.
And then last month, syncing my iPhone with iTunes inexplicably deleted the 40GB of my music from my iPhone. I hadn't done anything differently from the hundreds (?) of other times I've synced. I just had to go one-by-one and re-check all the playlists I sync, and wait a few hours for all the music to be re-copied. But still.
As easy as Apple products are to use, I'm always afraid something is going to go horribly wrong, in a way I never was with PC's, for example -- they were harder to use, and did less, but you could at least be reasonably sure of what was going on behind the scenes.
I've also never heard of them until now. Perhaps they should show up to said conferences, or at least keep tabs on Core Data-related talks at conferences and give the organizations/speakers a heads up.
As I understand it Simperium observes changes to the Core Data models you specify, maps those changes to operation transforms, and distributes the transforms across whichever clients have subscribed to the service. That's great if all your app needs is simple data replication to multiple clients. Unfortunately operational transforms are not necessarily snapshots of database transactions and conflict resolution strategies do not guarantee a valid database.
Saving a Core Data transaction on one client may generate multiple operations which are replayed interleaved with operations from other clients. Conflict resolution is then performed between operations rather than transactions. Simperium can probably take steps to minimize that case but as long as I can construct transactions which cannot be expressed as a single operation this will happen.
Worse a client can apply operations without conflicts only to find that its models no longer pass its own validation rules. For example two clients can each add a child object to a "has one" association. Both "create" operations can be applied without conflict but the parent object no longer passes the application's validation rules.
This doesn't make operational transform based synchronization untenable but it does imply a new set of constraints on how applications should model their data in order to successfully apply operations from other clients. That may not be a popular sales pitch but I'm reluctant to feed structured data through a service that doesn't at least discuss the model and its constraints in some detail. Otherwise I fear such a service is another iCloud like trap which will eventually (though perhaps less frequently) hit catastrophic edge cases from which there is little hope of recovery.
Interesting; I've implemented OT myself, and this is not a fundamental failing of the concept: in fact, even fairly simple implementations like ShareJS do not succumb to this fate (the easy way to model it is that you end up with larger macro-operations that are capable of transforming eachother's components, while still having to be played back one macro-op at a time; you also just stop playback while a transaction is in progress). I am totally willing to believe that Simperium's implementation (which I believe thinks about the separate objects as separately transformable documents) may have issues here, though.
> Worse a client can apply operations without conflicts only to find that its models no longer pass its own validation rules.
This is certainly true, but is endemic to the entire idea of offline synchronization and has nothing to do with operational transforms: you simply can't do this without accepting "a new set of constraints on how applications should model their data in order to successfully apply operations from other clients"; this is clear from CAP (as rather than become unavailable when the network is offline/partitioned, we have to lose some consistency).
> That may not be a popular sales pitch but I'm reluctant to feed structured data through a service that doesn't at least discuss the model and its constraints in some detail.
This is totally fair. (I can entirely appreciate it as well, as it reminds me of all of my complaints regarding how seldom you would hear companies like Parse and StackMob attempt to drill in how important security should be while using their services, at best leaving it as an appendix in their documentation. I believe they have at least been getting somewhat better about this since my talk on the subject at last year's 360|iDev.)
My hope is that now that one person (Jonathan Ive) has been put in charge of product development, things might get better.
It's a different game played by different rules.
Interestingly, I'd like to watch and see how Google does moving into the physical goods space.
For one, Windows Azure is a pretty big deal. Secondly, XBox Live is a pretty big deal. Third, Bing might be struggling to gain traction but it most certainly works.
So can you point me to a Microsoft web property or service that is as wonky as iCloud at all?
Dropbox does not come close as alternative. Dropbox as of now is file-based sync. Core Data iCloud sync is trying to sync an always opened SQLite3 database file, they do it through database log files. Possible alternatives are TouchDB, Simperium, Firebase but then you have to change your app from not using Core Data. Or rewrite your app to use file based storage as what 1password did. The key is offline sync. File based cloud sync is solved problem. So maybe the way to go is persistent store of client app has to use files.
Edit: regarding offline sync (which I didn't address) something like AFIncremental store is a good example of how a local persistent store can be maintained in tandem with the online store.
(But, as you say, non- trivial and hard to get right.)
https://github.com/stackmob/stackmob-ios-sdk/blob/master/Cor...
And once you discover that you can't fix all the problems that Apple is struggling with, you have dug yourself into a VERY deep hole :)
In my experience, it truly does "just work".
For developers, it's fundamentally broken as potatolicious describes https://news.ycombinator.com/item?id=5454729
There are other places where Dropbox has advantages over iCloud (like doucments being shared between applications, even though the dropbox api seem to move away from that and on to sandboxed application folders), but for the points made in the original article, Dropbox provides no help at all.
Further, by pushing an app relying on Dropbox, you're telling your users to sign up for an extra service to use your app. That's bad.
I'd never use it for a large team, but for a couple of collaborators, or syncing a repository between multiple computers for a single developer, it's a nice and easy solution.
Yes, sometimes I miss out on really neat developments. iCloud is one of those cases where I am glad for being a slow adopter. My mental red flags went up as soon as iCloud came out. Not one of my apps uses it.
On the end-user front iCloud is also an absolute mess. I probably have a dozen i-devices on the same account (testing, etc.). Crap appears and disappears from devices seemingly at random. The worst of it was what is happening with my wive's iPhone and mined. We share an iPad at home. Now contacts are getting deleted from her phone and mine at random and so are notes. It's at the point where she is not using notes any more because --her words-- they disappear within a hour or two of writing them.
Of course, turning off iCloud is the solution. The problem is that this isn't as simple as throwing the switch. If you turn off iCloud on your device it might actually delete things that it shoved into the iCloud service. And then it deletes the info from your other devices. And it does this in some amazingly weird ways. Without telling you what it is about to delete.
For example, I have never explicitly stored contact information on iCloud. Never. I type new or updated contact information into the phone. That's where it belongs. In fact, I never even use my iPad for this. Yet, some contact information for some strange fucking reason is sent over to iCloud. The first time I turned off iCloud piles of contact data evaporated from my phone. I had to turn the damn thing on again to get it back.
I haven't had the time to research how to fully disable iCloud on ALL of my devices while, at the same time, keeping or merging the state of my data on that particular device. As a developer I managed to avoid iCloud. As a user I was not that fortunate, I succumbed to the new shinny thing on the shelf. Now I want nothing to do with it. Thank you Apple.
Any ideas on how to do this?
Can anyone else chime in? Does this sound reasonable? I've had everything on my iPhone synced up to Google services for about 4 years now and it's never caused any issues.
Frankly, I'm just thinking of writing an app to export everything to a MySQL database on one of my servers and do my own version controlled backup. Not sure Apple would ever approve such an app, but I can certainly write it and use it internally.
A long time ago I learned there is nothing more important than ensuring your data is safe. I learned this the hard way in college when a drive failure evaporated six months of hard coding...and I had zero backups. You don't do something like that twice.
With iCloud Apple has given us a tool that is so badly done it has the potential to evaporate our data. Amazing.
There's a reason Microsoft code runs the IT infrastructures of small, medium and massive corporations. They get it. They don't engage in making petulant adolescent choices for their users. Crap like removing "Save As" from applications and forcing duplication --and even storage-- of files on iCloud from your desktop.
I've been using Apple products since the first Mac. I could not see any way to trust Apple with enterprise data. No way.
I really don't know what Apple is about any more.
We launched native iOS support a few weeks ago [2]. You can also access your data from JavaScript or from a REST API. Android and other platforms are in the works. We're a platform-independent service, and since you (the developer) is paying for the storage, we have the right incentives to give you maximum access to your data (unlike iCloud), and a good developer experience.
We've also put a ton of work into making it easy to learn and easy to use. See, for example, our quick tutorial [3].
If you've used Firebase as a substitute for iCloud, we'd appreciate your thoughts in the comments!
1. https://www.firebase.com/docs/transactions.html
2. https://www.firebase.com/blog/2013-03-20-new-firebase-ios-sd...
"If you turn off Documents and Data, all documents stored in iCloud will be deleted from this Mac"
Let's get past the jaw-to-the-floor, how-dare-you, !!!$$%#$## moment. They are morons and don't respect your data.
So, how do you figure out just what is in iCloud? I don't see an easy iCloud viewing option in Finder.
Doing some digging this is what I found:
- Open Finder
- Fire-up "Go To Tolder"
- Enter ~/Library/
- Go into "Mobile Documents"
That's what's on iCloud. You have to manually dig through it all and figure out what's there.If you delete anything there it will be deleted from all of your sync'ed devices, including your local Mac. It will also end-up in the trash folder.
You could copy it all to a USB drive, memory stick, or, better yet, Dropbox. Then you can completely disable iCloud knowing your data is safe and iCloud will never cause you further grief.
1 - http://www.tuaw.com/2011/06/24/the-icloud-logo-and-the-golde...
This was also when I learned that Google's process for restoring lost two-factor credentials ultimately ends in an email form which sends to higgins-prod-lost-hard-landing@google.com which been bouncing since 2011. Thank FSM I had some some backup codes printed…
iCloud would be worse if apps /couldn't/ opt out.
The scary thing is that you hope your apps can handle the case where they have local data but no authenticated user for it. Some of them just treat no authenticated user the same as a first start and reinit the local data.
Sometime last year I gave up waiting and started working on PouchDB (http://pouchdb.com/), its based on CouchDB's syncing model (and syncs with CouchDB) and works anywhere that Javascript runs (the browser + node), there is an iOS / OSX equivalent (https://github.com/couchbaselabs/TouchDB-iOS).
PouchDB isnt quite ready for production but its getting close, I would love any feedback on it, although I expect most people commenting on this would be more interested in TouchDB
How about the initial replication of larger databases? Do you roll out an initial version directly in your app or do you have some file based download of a snapshot?
I used to work for Couchbase on the arm compiled couchdb mobile versions, mainly android but either way I wouldnt hugely recommend using them, they always worked against the grain of the platform, never had great performance and due to those reasons they didnt really get widely adopted and therefore I wouldnt trust their stability across a variety of platforms and use cases.
The problems with arm compiled couchdb is the reason TouchDB was made, and similiarly with PouchDB / TouchDB for android and CouchDB I hope we get to the point were there is an out of the box sync solution that works great on every platform, it doesnt feel too far away.
For large initial datasets I have tried taking a dump of the database and importing it on first load, it worked however replication has been more than sufficient for my use cases right now. Once pouch has stabilised and released there are a lot of low hanging fruit that can be done to optimise replication (and in particular for first load).
https://github.com/couchbaselabs/TouchDB-iOS
Of course, you would need to support some form of CouchDB server for this to work, deal with authentication, etc.
Reading documents from the local TouchDB store would block the main thread until the upstream CouchDB server would reply with the document's current revision, even if the local store already has an existing version of the document. Using GCD to wrap around these document transactions was still clunky at best and resulted in a poor overall UX.
YMMV, perhaps these issues could have been worked around if I had had a more realistic deadline and stayed in touch with the community. The maintainer, Jens Alfke, is very active.
It's like Game Center, Apple doesn't bother with any apps that really use it so there's many, many problems reported on a consistent basis, and they don't seem to use iCloud for anything involving Core Data so they're not dog fooding and having engineers swear at code. No swearing means no one is motivated to go back and try improve it.
They, understandably, thought there was no way that something so heavily promoted by Apple could be such a lemon. And in spite of having heard mumblings of iCloud bugs, we wrote it off as amateur iOS developers.
We ultimately abandoned iCloud prior to release (going with simple Python based server for data storage) -- reading the other posts I'm glad we didn't try to convince ourselves that the 'almost stable' behavior was just a quirk of our devices having been used in development mode.
How can the richest computing company not answer questions put by thousands of users who have banged their head in the same bug? It's insulting to have this circus happening on their doorstop.
[1]http://www.theverge.com/2013/3/26/4148628/why-doesnt-icloud-...
If I have many apps that do the same thing, I'd have to go through each one to remember where I saved something. No, just no.
Serious Apple have always done a crappy job of online services - why would you expect any more.