I'm not so bugged by this. Uploading data to iCloud has always been a trade of convenience at the expense of privacy. Adding a client-side filter isn't great, but it's not categorically unprecedented--Apple executes search warrants against iCloud data--and can be turned off by turning off iCloud back-ups.
The scanning of childrens' iMessages, on the other hand, is a subversion of trust. Apple spent the last decade telling everyone their phones were secure. Creating this side channel opens up all kinds of problems. Having trouble as a controlling spouse? No problem--designate your partner as a child. Concerned your not-a-tech-whiz kid isn't adhering to your house's sexual mores? Solved. Bonus points if your kid's phone outs them as LGBT. To say nothing of most sexual abuse of minors happening at the hands of someone they trust. Will their phone, when they attempt to share evidence, tattle on them to their abuser?
Also, can't wait for Dads' photos of their kids landing them on a national kiddie porn watch list.
I'm not a parent but the other child protection features seem like they could definitely be abused by some parents to exert control/pry into their kids private lives. It's a shame that systems have to be designed to prevent abuse by bad people but at Apple's scale it seems like they should have better answers for the concerns being raised
It would be easy to extend this to scan for 'wrongthink'.
Next logical steps would be to scan for: confidential government documents, piracy, sensitive items, porn in some countries, LGBT content in countries where it's illegal, etc... (and not just on icloud backed up files, everything)
This could come either via Apple selling this as a product or forced by governments...
Next it’s Covid misinformation
Then eventually they’re coming for your Bernie memes
Well the obvious response is that these systems don't have to be designed. Child abuse is a convenient red herring to expand surveillance capabilities. Anyone opposing the capability is branded a child molester. This is the oldest trick in the book.
I mean the capability to spy on your kid can easily be used to abuse them. Apple could very well end up making children's lives worse.
It seems the only way to opt-out is to get out of the Apple ecosystem.
https://www.golem.de/news/hamburg-polizei-nutzt-corona-konta...
https://www.ccc.de/de/updates/2021/luca-app-ccc-fordert-bund...
That's the irony in this: This move arguably improves privacy by removing the requirement that images be decrypted on the server to run a check against the NCMEC database. While iCloud Photo Library is of course not E2E, in theory images should no longer have to be decrypted anywhere other than on the client under normal circumstances.
And yet – by moving the check to the client, something that was once a clear distinction has been blurred. I entirely understand (and share) the discomfort around what is essentially a surveillance technology now running on hardware I own rather than on a server I connect to, even if it's the exact same software doing the exact same thing.
Objectively, I see the advantage to Apple's client-side approach. Subjectively, I'm not so sure.
It’s a huge move, and a big change in the presumptions of how their platform works.
I’m heavily invested in the Apple ecosystem and it’ll take a years work to get off it.
I’m thinking of the prevailing principles of whatever I do best, and one of them is, excise integrated platforms as much as possible.
But most consumers will probably not care, and this road will get paved for miles to come.
That is good, but unless a system like this is fully open source and runs only signed code there really aren't many protections against abuse.
This will be the statement they increase the scope the next time.
Hard to imagine that Tim Cook would have scanned Epsteins photos...
That's not how it works, unless you control your partner's Apple ID and you lie about their DOB when you create their account.
I created my kids Apple IDs when they were minors and enrolled them in Family Sharing. They are now both over 18 and I cannot just designate them as minors. Apple automatically removed my ability to control any aspects of their phones when they turned 18.
> Dads' photos of their kids landing them on a national kiddie porn watch list.
Indeed, false positives is much more worrying. The idea that my phone is spying on my pictures... like, what the hell.
Rather than reassuring me, this sounds like an achievable set of steps for an abuser to carry out.
There’s a repository built from seized child porn.
Those pictures and videos have hashes. Apple wants to match against those hashes.
That’s it.
That’s it for now.
I don't know how most people feel about this, but even a false positive would seem hazardous. Does that put you on some permanent watch list in the lowest tier? How can you even know? And besides, it's all automated.
We could of course massively shift society towards a no-photo/video policy for our kids (perhaps only kept on a non-internet connected camera and hard drive), and tell grandma to just deal with it (come back after the lockdown granny, if you survive). Some people do.
And don't think that normal family photos won't get classified as CEI. What is titillating for one is another's harmless family photo.
Previous times down this slope:
* UK internet filters for child porn -> opt out filters for regular porn (ISPs now have a list of porn viewers) + mandatory filters for copyright infringment
* Google drive filters for illegal content -> Google driver filters for copyrighted content
* iCloud data is totally protected so it's ok to require an apple account -> iCloud in China run by government controlled data centers without encryption
* Protection against malware is important so Windows defender is mandatory unless you have a third party program -> Windows Defender deletes DeCSS
* Need to protect users against malware, so mobile devices are set up as walled gardens -> Providers use these walled gardens to prevent business models that are bad for them
But just to highlight one aspect, the list of maintained hashes has a known, non-negligible fraction of false positives.
> That’s it for now.
If this is an attempt at "first they came...", we're not biting.
Well, as a parent, I can tell you that some 16/17 year olds are responsible and worthy of the trust that comes with full independence. Others have more social/mental maturing to do yet and need some extra guidance. That's just how it goes.
Register your kids with a corporate behemoth! Why not!? Get them hooked on Apple right from childhood, get their entire life in iCloud, and see if they'll ever break out of the walled garden.
This is an argument for me to not start using iCloud keychain. If Apple flags my account, I don't want to lose access to literally all my other accounts.
Also, a father would only land in a national registry of their child’s photos are known to be CSAM. Simply taking a photo of your child wouldn’t trigger it.
The most annoying thing about Apple Family sharing is that in order to create accounts for people you must specify that they are under 13 (source: https://www.apple.com/lae/family-sharing) - otherwise the only other option is for your "family member" to link their account to the Apple Family which is under your purview, which understandably many people might be hesitant to do because of privacy concerns (as opposed to logging into the child account on a Windows computer exclusively to listen to Apple Music - which doesn't tie the entire machine to that Apple ID as long as it's not a mac).
And so in my case, I have zero actual family members in my Apple Family (they're more interested in my Netflix family account). It begs the question, why does Apple insist on having people be family members in order to share Apple Music? We have five slots to share, and they get our money either way. They also don't let you remove family members - which may be the original intent for insisting on such a ridiculous thing - as if they're trying to take the moral high ground and guilt trip us for disowning a family member when in fact it simply benefits them when a fallout occurs between non-family members, because there's a good chance that the person in question will stop using the service due to privacy concerns, and that's less traffic for Apple.
It's actually kind of humorous to think that I still have my ex-ex-ex-girlfriend in my Apple Family account, and according to Apple she's 11 now (in reality, she's in her 30s). I can't remove her until another 7 years pass (and even then it’s questionable if they’ll allow it, because they might insist that I can’t divorce my “children”). And honestly, at this point I wouldn’t even remove her if I could, she has a newborn baby and a partner now, and I’m happy to provide that account, and I still have two unused slots to give away. I’ve never been the type of person who has a lot of friends, I have a few friends, and one girlfriend at a time. But the thing is she’s never been a music person and I assume that she isn’t even using it - and so even if I made a new best friend or two and reached out to her to let her know that I wanted to add them, Apple currently wouldn’t let me remove her to make room for those theoretical friends. While I'm a big fan of Apple hardware, it really bothers me that a group of sleazy people sat around a table trying to figure out how to maximize income and minimize network traffic, and this is what they came up with.
There’s two functions here. Both client side.
First, machine learning to detect potentially inappropriate pictures for children to view. This seems to require parental controls to be on. Optionally it can send a message to the parent when a child purposefully views the image. The image itself is not shared with Apple so this is notification to parents only.
The second part is a list of hashes. So the Photos app will hash images and compare to the list in the database. If it matches then presumably they do something about that. The database is only a list of KNOWN child abuse images circulating.
Now, not to say I like the second part but the first one seems fine. The second is sketchy in that what happens if there’s a hash collision. But either way it seems easy enough to clear that one up.
No father is going to be added to some list for their children’s photos. Stop with that hyperbole.
"But it only impacts iCloud Photos". Valid! So why not run the scanner in iCloud and not on MY PHONE that I paid OVER A THOUSAND DOLLARS for? Because of end-to-end encryption. Apple wants to have their cake and eat it too. They can say they have E2EE, but also give users no way to opt-out of code, running on 100% of the "end" devices in that "end-to-end encryption" system, which subverts the E2EE. A beautiful little system they've created. "E2EE" means different things on Apple devices, for sure!
And you're ignoring (or didn't read) the central, valid point of the EFF article: Maybe you can justify this in the US. Most countries are far, far worse than the US when it comes to privacy and human rights. The technology exists. The policy has been drafted and enacted; Apple is now alright with subverting E2EE. We start with hashes of images of child exploitation. What's next? Tank man in China? Photos of naked adult women, in conservative parts of the world? A meme criticizing your country's leader? I want to believe that Apple will, AT LEAST, stop at child exploitation, but Apple has already estroyed the faith I held in them, only yesterday, in their fight for privacy as a right.
This isn't an issue you can hold a middleground position on. Encryption doesn't only kinda-sorta work in a half-ass implementation; it doesn't work at all.
I am wondering what hashes are now and will be in this database. Or combine Pegasus exploit , put a few bad images on the journalist/politician iPhone, cleanup the tracks and wait for Apple and FBI destroy the person.
I kept the lineage phone in my back pocket, confident that it would be a good 4-5 years before they shipped something that violated their claims. I figured, the alternatives would be stable and widespread.
My timing was off.
This of course gets into 'what even is harm?' since that's a very subjective way of classifying something, especially when you try to do it on behalf of others.
For CSAM you could probably assume that "everyone this code takes action against would consider doing so harmful", but _consequences in general are harmful_ and thus you could make this same argument about anything that tries to prevent crime or catch criminals instead of simply waiting for people to turn themselves in. You harm a burglar when you call for emergency services to apprehend them.
> This isn't an issue you can hold a middleground position on. Encryption doesn't only kinda-sorta work in a half-ass implementation; it doesn't work at all.
This is the exact issue that the U.S. has been entrenched by - thinking that you can't disagree with one thing someone says or does and agree with other things they say or do. You can support Apple deciding to combat CSAM. You can not support Apple for trying to do this client-sided instead of server-sided. You can also support Apple for taking steps towards bringing E2EE to iCloud Photos. You can also not support them bowing to the CCP and giving up Chinese citizens' iCloud data encryption keys to the CCP. This is a middle ground - and just because you financially support Apple by buying an iPhone or in-app purchases doesn't mean you suddenly agree with everything they do. This isn't a new phenomenon - before the internet, we just didn't have the capacity to know, in an instant, the bad parts of the people or companies we interfaced with.
I've seen the "right to unreasonable search and seizure" Americans hold quoted a bit during this discussion. Valid, though to be clear, the Constitution doesn't apply for private company products. But more interestingly: what about right against self-incrimination? That's what Apple is pushing here; that by owning an iPhone, you may incriminate yourself, and actually it may end up happening whether you're actually guilty or not.
Um. No?
I would be very surprised if more than 10% of people in possession of sexual images of under 18s molested (pre-pubecent) children.
I prefer not having my door busted in at 6 am and my dog shot in the head because of a hash collision
https://www.aclu.org/gallery/swat-team-blew-hole-2-year-old-...
PS. On a personal note, Apple is done for me. Stick a fork in it. I was ready to upgrade after September especially since I heard touch-ID is coming back and I love my iPhone 8. But sure as hell this sad news means i8 is my last Apple device.
This seems fine as it's (a) being done on iCloud-uploaded photos and (b) replacing a server-side function with a client-side one. If Apple were doing this to locally-stored photos on iCloud-disconnected devices, it would be nuts. Once the tool is built, expanding the database to include any number of other hashes is a much shorter leap than compelling Apple to build the tool.
> it seems easy enough to clear that one up
Would it be? One would be starting from the point of a documented suspicion of possession of child pornography.
Ive actually witnessed someone go through this from someone else getting caught with these types of images and attempting to bring others with him. It’s not easy. It took him over a year of his life calling constantly asking when the charges will be dropped. They even image your devices on the spot yet still take them and stuff them in an evidence locker until everything is cleared up. You’re essentially an outcast to society while this is pending as well as most people assume if you have police interest related to child pornography you must be guilty.
I’d be happier if Apple wasn’t doing this at all. I’m not defending them necessarily but I am calling bullshit on your scare tactics. It’s not necessary.
Until they push a small change to the codebase...
@@ -7637,3 +7637,3 @@
-if (photo.isCloudSynced && scanForIllegalContent(photo)) {
+if (scanForIllegalContent(photo)) {
reportUserToPolice();
} -if (photo.isCloudSynced && scanForIllegalContent(photo)){
+if (photo.isCloudSynced & scanForIllegalContent(photo)) {
reportUserToPolice();
}
So it would be better to say "this has been a risk by using modern technology".
Even if everything is imperfect, some things are more imperfect than others. If each component that makes you vulnerable has a given percent chance of being used against you in practice, you're better off with one than six, even if you're better off with none than one.
Moreover, having a personal route to digital autonomy is nearly worthless. To protect democracy and freedom, practically all users need to be able to compute securely.
For example the only closed source component that I use is the NVIDIA driver, but I could use Nouveau, with lower performance.
The real problem is caused by the hardware backdoors that cannot be controlled by the operating systems and which prevent the full ownership of the devices, e.g. the System Management Mode of Intel/AMD, the Intel ME and the AMD PSP.
And let's not forget the minor detail that this is now public knowledge. It's like telling your teenage son you're going to be searching his closet for marijuana in the future.
GoodReader has optional app-level file encryption with a password that is not stored in the iOS keychain. In theory, those encrypted files should be opaque to device backups or local filesystem scanning, unless iOS or malware harvests the key from runtime memory, https://goodreader.com/
This is way too much work to gain hardly anything. It's just as easy to just log into another device with their iCloud password and literally read everything they send. Less work, more result.