Fortunately, my email is on a paid provider (fastmail), and my photos are on a NAS, I've worked hard to get all of my friends on Signal. While I still use google maps, I've been trialing out OSM alternatives for a minute.
The things they've described are in general, reasonable and probably good in the moral sense. However, I'm not sure that I support what they are implementing for child accounts (as a queer kid, I was terrified of my parents finding out). On the surface, it seems good - but I am concerned about other snooping features that this portents.
However, with icloud photos csam, it is also a horrifying precedent that the device I put my life into is scanning my photos and reporting on bad behavior (even if the initial dataset is the most reprehensible behavior).
I'm saddened by Apple's decision, and I hope they recant, because it's the only way I will continue to use their platform.
I'm not so bugged by this. Uploading data to iCloud has always been a trade of convenience at the expense of privacy. Adding a client-side filter isn't great, but it's not categorically unprecedented--Apple executes search warrants against iCloud data--and can be turned off by turning off iCloud back-ups.
The scanning of childrens' iMessages, on the other hand, is a subversion of trust. Apple spent the last decade telling everyone their phones were secure. Creating this side channel opens up all kinds of problems. Having trouble as a controlling spouse? No problem--designate your partner as a child. Concerned your not-a-tech-whiz kid isn't adhering to your house's sexual mores? Solved. Bonus points if your kid's phone outs them as LGBT. To say nothing of most sexual abuse of minors happening at the hands of someone they trust. Will their phone, when they attempt to share evidence, tattle on them to their abuser?
Also, can't wait for Dads' photos of their kids landing them on a national kiddie porn watch list.
I think many queer people have a completely different idea of the concept of "why do you want to hide if you're not doing anything wrong" and the desire to stay private. Especially since anything sexual and related to queerness is way more aggressively policed than hetero-normative counterparts.
Anything "think of children" always has a second order affect of damaging queer people because lots of people still think of queerness as dangerous to children.
It is beyond likely that lots of this monitoring will catch legal/safe queer content - especially the parental-controls focused monitoring (as opposed to the gov'ment db of illegal content)
How, how is it even morally good?? Will they start taking pictures of your house to see if you store drugs under your couch? Or cook meth in your kitchen??
What is moral is for society to be in charge of laws and law enforcement. This vigilante behavior by private companies who answer to no one is unjust, tyrannical and just plain crazy.
We've collectively handed control of our personal computing devices over to Apple and Google. I fear the long-term consequences of that will not be positive...
Its been clear Tim Cook was going to slowly harm the brand. He was a wonderful COO under a visionary CEO-type, but he holds no particular "Tech Originalist" vision. He's happy to be part of the BigTech aristocracy, and probably feels really at home in the powers it affords him.
Anyone who believes this is "just about the children" is naive. His chinese partners will use this to crack down on "Winnie the Poo" cartoons and the like...before long questioning any Big Pharma product will result in being flagged. Give it 5 years at max.
See the comment threads around this topic, and look back to other related events (notably the tech giants censoring people "for the betterment of society" in the past 12 months).
Boiling a frog may happen slowly, but the water continues to heat up even if we pretend it doesn't. Very disappointed with this action by Apple.
Apple thinks the appropriate time for queer kids to find themselves is after they turn 18.
You can still use Google Maps without an account and "incognito". I wish they'd allow app store usage without an account though- similar to how any Linux package manager works.
Apple seems to not have interest in users devices, which makes sense -- they're not liable for them. They _do_ seem interested in protecting the data that they house, which makes sense, because they're liable for it and have a responsibility to remove/report CSAM that they're hosting.
That precedent was set many years ago.
>a man [was] arrested on child pornography charges, after Google tipped off authorities about illegal images found in the Houston suspect’s Gmail account.
Microsoft’s “PhotoDNA” technology is all about making it so that these specific types of illegal images can be automatically identified by computer programs, not people.
PhotoDNA converts an image into a common black-and-white format and size the image to a uniform size, Microsoft explained last year while announcing its increased efforts at collaborating with Google to combat online child abuse.
https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-le...
What I always thought was interesting was that the Police Security Services in Singapore were called "CISCO" -- and you used to see these swat-APV-type vans driving around and armed men with CISCO emblazened on their gear/equip/vehicles...
I always was reminded of Cyberpunk Anime around that.
But if your worry is governments reading your mail, is an email company any safer? I’m sure FM doesn’t want to scan your mail for the NSA or its Australian proxy, but do they have a choice? And if they were compelled, would they not be prevented from telling you?
“We respect your privacy” is exactly what Apple has been saying.
Have you tried using the website? I've had some luck with that on postmarketOS, and it means you don't need to install Play services to use it.
I use Citymapper simply because I find it better (for the city-based journeys that are my usual call for a map app) - but it not being a Google ~data collection device~ service is no disadvantage.
At least, depending why you dislike having everything locked up with Google or whoever I suppose. Personally it's more having everything somewhere that troubles me, I'm reasonably happy with spreading things about. I like self-hosting things too, just needs a value-add I suppose, that's not a reason in itself for me.
Is there a way to set up Android to handle shared locations without Google Maps?
Every time someone shares location with me (in Telegram) it displays as a tiny picture and once I click it it says I have to install Google Maps (I use an alternative for actual maps and don't have Google Maps installed). So I end up zooming the picture and then finding the location on the map manually.
apples new customers are the various autocratic regimes that populate the earth. apples customers used to be human beings. there exist many profiteers in mountain view, cupertino menlo and atherton in the service of making our monopolies more capable of subjugating humanity.
So I don't actually expect my mail to be private. But at least it's not Google.
This is very much like driving a car through a crowd of protestors. They will slowly, inexorably, eventually push through.
Paid doesn't mean more secure, it's popular mistake.
If you don't want your parents to look at your phone, you shouldn't be using a phone owned by your parent's account. The new feature doesn't change this calculus.
As a queer kid, would you enjoy being blackmailed by someone who tricked you into not telling your parents?
I bought an iPhone because the CEO seemed to be sincere in his commitment to privacy.
What Apple has announced here seems to be a complete reversal from what I understood the CEO saying at the conference only a few years ago.
https://twitter.com/josephfcox/status/1423382200880439298/ph...
- It's run for Messages in cases where a child is potentially viewing material that's bad.
- It's run _before upload to iCloud Photos_ - where it would've already been scanned anyway, as they've done for years (and as all other major companies do).
To me this really doesn't seem that bad. Feels like a way to actually reach encrypted data all around while still meeting the expectations of lawmakers/regulators. Expansion of the tech would be something I'd be more concerned about, but considering the transparency of it I feel like there's some safety.
https://www.apple.com/child-safety/ more info here as well.
Hanlon's razor does not apply to megacorporations that have enormous piles of cash and employ a large number of very smart people, who are either entirely unscrupulous or for whom scruples are worth less than their salaries. We probably aren't cynical enough.
I am not arguing that we should always assume every change is always malicious towards users. But our index of suspicion should be high.
> Andrew Stone, who worked with Jobs for nearly 25 years, told the site Cult of Mac last week that Steve Jobs resisted letting Apple be part of PRISM, a surveillance program that gives the NSA access to records of major Internet companies. His comments come amid speculation that Jobs resisted cooperating. “Steve Jobs would’ve rather died than give into that,” Stone told the site.
> According to leaked NSA slides about PRISM, Apple was the last tech behemoth to join the secret program — in October 2012, a year after Jobs died. Apple has said that it first heard about PRISM on June 6 of this year, when asked about it by reporters.
https://www.huffpost.com/entry/apple-nsa-steve-jobs_n_346132...
I mean, maybe they didn't call it "PRISM" when talking about it with Cook, so it could technically be true that they didn't hear of PRISM until media stories. Everyone knows the spy agency goes around telling all of their project code names to companies they're trying to compromise. Hello, sir. We're here to talk to you about our top secret surveillance program we like to call PRISM where we intercept and store communications of everyone. Would you like to join? MS did. So did Google. Don't you want to be in our select cool club?
My biggest turning point was Tim Cook flat out lying on the Apple case against Qualcomm. Double Dipping? Qualcomm patents being more than double than all the other six combined? And the tactics they used in court which was vastly different to Apple vs Samsung's case. And yes, they lost. ( Or settled )
That is the same with privacy. They simplifies their PR message as tracking = evil. Tracking is invading your privacy. Which is all good. But at the same time Apple is tracking you, everything you do on Apple Music, Apple TV+, App Store and even Apple Card. ( They only promise not to sell your Data to third party, they still have some of those Data. ). What that means is that only Apple is allowed to track you, but anyone else doing it are against privacy? What Apple really meant by the word Privacy then is that Data should not be sold to third parties. But no, they intentionally keep it unclear and created a war on Data Collection while they are doing it. And you now have people flat out claiming Apple doesn't collect any Data.
Then there is a war on Ads. Which was so bad the Ad industry pushes back and Tim Cook had to issue a mild statement saying they are not against Ads, only targeted Ads. What?
Once you start questioning all of his motives, and find concrete evidence that he is lying, along with all the facts from court case of how Apple has long term plans to destroy other companies, they all line up and shape how you view Tim Cook's Apple. And it isn't pretty.
And that is speaking from an Apple fan for longer than two decade.
For all the rhetoric about privacy coming from Apple, I feel that such an extreme measure would surely cause complaints from anyone deeply invested in privacy. And maybe they're just using words like "significant privacy benefits compared to previous techniques" to make it sound reasonable to the average user who's not that invested in privacy.
The sincerity of a company officer, even the CEO, should not factor into your assessment. Officers change over time (and individuals can change their stance over time), after all.
It was back when Apple had just introduced the (now-abandoned) Force Touch feature (i.e., pressure sensitive touch, since abandoned, since it turns out pushing hard on an unyielding surface is not very pleasant or useful).
To showcase the capability, Apple had updated many of its apps with new force-touch features. One of which was mail: if you pushed just right on the subject line of a message, you'd get a tiny, unscrollable popout preview of its contents.
It was totally useless: it took just as much time to force touch to see the preview as just normally tapping to view the message, and the results were less useful. It was also fairly fiddly: if you didn't press hard enough, you didn't get the preview; if you pressed too hard, it would open into the full email anyway.
So Tim Cook, demoing the feature, said a funny thing. He said, "It's great, I use it all the time."
Which maybe, just maybe, is true, but personally I don't believe, not for a second.
So since then, I've had Tim down in my book as basically a big liar.
I'm in shock. Multi-billion dollars company usually never lies to make money! And power grabbing entities have such a neat track record in human history.
Not to mention nobody saw that coming and told repeatadly one should not get locked into such a closed and proprietary ecosystem in the first place.
I mean, dang, this serial killer was such a nice guy. The dead babies in the basements were weird but appart from that he was a stellar neighbour.
Apple is moving that process from on server to on phone in a way that protects your privacy better than current standards.
In the current system, all your photos are available to Apple unencrypted. In the new system, nothing will be visible to apple unless you upload N images with database hits. From those N tokens, Apple is then able to decrypt your content.
So when this feature lands, it improves your privacy relative to today.
1. PhotoDNA is already scanning content from Google Photos and a whole host of other service providers.
2. Apple is obviously under pressure to follow suit, but they developed an on-device system, recruited mathematicians to analyze it, and published the results, as well as one in-house proof and one independent proof showing the cryptographic integrity of the system.
3. Nobody, and I mean nobody, is going to successfully convince the general public that a tool designed to stop the spread of CSAM is a "bad thing" unless they can show concrete examples of the abuse.
For one and two: given the two options, would you rather that Apple implement serverside scanning, in the clear, or go with the on-device route? If we assume a law was passed to require serverside scanning (which could very well happen), what would that do to privacy?
For three: It's an extremely common trope to say that people do things to "save the children." Well, that's still true. Arguing against a CSAM scanning tool, which is technically more privacy preserving than alternatives from other cloud providers, is an extremely uphill battle. The biggest claim here is that the detection tool could be abused against people. And that very well may be possible! But the whole existence of NCMEC is predicated on stopping the active and real danger of child sex exploitation. We know with certainty this is a problem. Compared to a certainty of child sex abuse, the hypothetical risk from such a system is practically laughable to most people.
So, I think again, the backlash is daft. It's been about two days of the announcement being public (leaks). The underlying mathematics behind the system has barely been published [0]. It looks like the EFF rushed to make a statement here, and in doing so, it doesn't look like they took the time to analyze the cryptography system, to consider the attacks against it, or to consider possible motivations and outcomes. Maybe they did, and they had advanced access to the material. But it doesn't look like it, and in the court of public opinion, optics are everything.
[0]: https://www.apple.com/child-safety/pdf/Alternative_Security_...
It's certainly said to be designed to do it, but have you seen concerns raised in the other thread (https://news.ycombinator.com/item?id=28068741)? There have been reports from some commenters of the NCMEC database containing unobjectionable photos because they were merely found in a context alongside some CSAM.
Who audits these databases? Where is the oversight to guarantee only appropriate content is included? They are famously opaque because the very viewing of the content is illegal. So how can we know that they contain what they are purported to contain?
This is overreach.
As with all politically-motivated initiatives that boldly violate the Constitution (consider the FISA Court, and its rubber stamp approval of 100% of the secret warrants put before it), the use and abuse of this system will go largely underground, like FISA, and its utility will slowly degrade due to lack of oversight. In time, even bad matches will log the IDs of both parties in databases that label them as potential sexual predators.
Believe it. That's how modern computer-based gov't intel works. Like most law enforcement policy recommendation systems, Apple's initial match algorithm will never be assessed for accuracy, nor be accountable for being wrong at least 10% of the time. In time it will be replaced by other third party screening software that will be even more poorly written and overseen. That's just what law enforcement does.
I've personally seen people suffer this kind of gov't abuse and neglect as a result of clueless automated law enforcement initiatives after 9-1-1. I don't welcome more, nor the gradual and willful tossing of everyone's basic Constitutional rights that Apple's practice portends.
The damages to personal liberty that are inherent in conducting secret searches without cause or oversight is exactly why the Fourth Amendment requires a warrant before conducting a search. NOW is the time to disabuse your sense of 'daftness'; not years from now, after the Fourth and Fifth Amendments become irreversibly passe. Or should I say, 'daft'?
Apple employs cryptographers, but they are not necessarily acting in your interest. Case in point: their use of private set intersection, to preserve privacy..of law enforcement, not users. Their less technical summary:
> Instead of scanning images in the cloud, the system performs on-device matching using a database of known CSAM image hashes provided by NCMEC and other child safety organizations. Apple further transforms this database into an unreadable set of hashes that is securely stored on users’ devices.
> Before an image is stored in iCloud Photos, an on-device matching process is performed for that image against the known CSAM hashes. This matching process is powered by a cryptographic technology called private set intersection..
The matching is performed on device, so the user’s privacy isn’t at stake. But, thanks to PSI and the hash preprocessing, the user doesn’t know what law enforcement is looking for.
> I think again, the backlash is daft.
Don't apologize for this bullshit! Don't let your love of brand trump the reality of what's going on here.
Machinery is being put in place to detect what files are on your supposedly secure device. Someone has the reins and promises not to use it for anything other than "protecting the children".
How many election cycles or generations does it take to change to an unfavorable climate where this is now a tool of great asymmetrical power to use against the public?
What happens when the powers that be see that you downloaded labor union materials, documents from Wikileaks, or other files that implicate you as a risk?
Perhaps a content hash on your phone puts you in a flagged bucket where you get pat downs at the airport, increased surveillance, etc.
The only position to take here is a full rebuke of Apple.
edit: Apple apologists are taking a downright scary position now. I suppose the company has taken a full 180 from their 1984 ad centerpiece. But that's okay, right, because Apple is a part of your identity and it's beyond reproach?
edit 2: It's nominally iCloud only (a key feature of the device/ecosystem), but that means having to turn off a lot of settings. One foot in the door...
edit 3: Please don't be complicit in allowing this to happen. Don't apologize or rationalize. This is only a first step. We warned that adtech and monitoring and abuse of open source were coming for years, and we were right. We're telling you - loudly - that this will begin a trend of further erosion of privacy and liberty.
TIL - (2014) PhotoDNA Lets Google, FB and Others Hunt Down Child Pornography Without Looking at Your Photos
https://petapixel.com/2014/08/08/photodna-lets-google-facebo...
Fighting to preserve a freedom is not daft, even if it is David vs. Goliath's bigger, meaner brother and his friends.
Would it be ok to use this approach to stop "terrorism"? Are you ok with both Biden and Trump defining that list?
They should just use their own computers to do this stuff.
Apple remains the most privacy respecting major vendor. The only way to do better is fully open software and open hardware.
I've had enough of the "think of the children" arguments.
This is the perfect way to begin opening the backend doors.
I’m considering a 24h black out with a protest link to apple’s support email explaining what they’ve done.
I wonder if anyone else would join me?
Russian and ex-Soviet countries with human trafficking mafias host several fucked up people who produce this crap.
So every iPhone will now host the explicit images from the National Center for Missing & Exploited Children database.
Source (PDF): https://www.apple.com/child-safety/pdf/Technical_Assessment_...
It's hashes, not the images themselves.
> If an account held by a child under 13 wishes to send an image that the on-device machine learning classifier determines is a sexually explicit image, a notification will pop up, telling the under-13 child that their parent will be notified of this content. [...] For users between the ages of 13 and 17, a similar warning notification will pop up, though without the parental notification.
Why is it different for children under 13, specifically? The 18-year cutoff makes sense, because turning 18 carries legal weight in the US (as decided via a democratic process), but 13?
13 is an age when many parents start granting their children more freedom, but that's very much rooted in one's individual culture—and the individual child. By giving parents fewer options for 13-year-olds, Apple—a private company—is pushing their views about parenting onto everyone else. I find that a little disturbing.
---
Note: I'm not (necessarily) arguing for greater restrictions on 13-year-olds. Privacy for children is a tricky thing, and I have mixed feelings about this whole scheme. What I know for sure, however, is that I don't feel comfortable with Apple being the one to decide "this thing we've declared an appropriate invasion of privacy for a 12-year-old is not appropriate for a 13-year-old."
- Apple sends Bob’s info to law enforcement; Bob is swatted or his life is destroyed in some other way. Worst, but most likely outcome.
- An Apple employee (or an outsourced contractor) reviews the photo, comparing it to CSAM source image sample used for the hash. Only if the image matches according to human vision, Bob is swatted. This requires there to be some sort of database of CSAM source images, which strikes me as unlikely.
- An Apple employee or a contractor reviews the image for abuse without comparing it to CSAM source, using own subjective judgement. Better, but implies Apple employees could technically SWAT Apple users.
e: It is definitely not a strict/cryptographic hash algorithm: "Apple says NeuralHash tries to ensure that identical and visually similar images — such as cropped or edited images — result in the same hash." They are calling it "NeuralHash" -- https://techcrunch.com/2021/08/05/apple-icloud-photos-scanni...
Downloaded images are not uploaded to iCloud w/out user intervention.
> Apple says that it will manually review each report to confirm there is a match. It can then take steps to disable a user's account and report to law enforcement.
So at least it’s the last option.
In your house, you might have private documents, do some things you don't want other people to have or see just like what we have on our phones nowadays.
The analogy I'm trying to make is that if suddenly the government decided to install cameras in every houses with the premise to make sure no pedophile is abusing a child and that the cameras never send data unless the AI done locally detects it is something that I believe would shock everyone.
unfortunately the law hasn't really kept up with technology. Let's hope this gets in front of a judge who's able to extrapolate some 'digital' rights from the (outdated) constitution. Unless of course they also 'think of the children'.
it doesn't help that most things online are very abstract, with terms like 'the cloud' making things even harder to understand, which in reality is just someone else's computer
How do we know Apple or the FBI don’t do this? If they want to search someone’s phone all they need to do is enter a hash of a photo they know is on the targets phone and voila, instant access.
Also, how is this not a violation of the 14th amendment? I know Apple isn’t part of the government but they are basically acting as a defacto agent of the police by scanning for crimes. Using child porn as a completely transparent excuse to start scanning all our material for anything they want makes me very angry.
Because it requires Apple and law enforcement, two separate organizations, to collude against you.
The false positive would have to be affirmed to a court and entered into evidence. If the false positive we’re found to not match the true image by the court, any warrant etc. would be found invalid and the fruit of any search etc would be invalid as well.
Apple is a private company. By agreeing to use iCloud photos you agree to their terms, this no 14th amendment violation.
All one needs to do, in order to flag someone or get them caught up in this system, is to gain access to this list of hashes and construct an image. This data is likely to be sought after as soon as this system is implemented, and it will only be a matter of time before a data breach exposes it.
Once that is done, the original premise and security model of the system will be completely eroded.
That said, if this does get implemented I will be getting rid of all my Apple devices. I’ve already switched to Linux on my development laptops. The older I get, the less value Apple products have to me. So it won’t be a big deal for me to cut them out completely.
https://en.wikipedia.org/wiki/Cryptographic_hash_function
that said, it's not clear to me from
https://www.apple.com/child-safety/pdf/Apple_PSI_System_Secu...
how collision resistant what's to be used will be.
if you really want to save the children, why not build the scanning into safari? scan the whole phone! just scan it all. its really no different than what they are doing. its not like they would have to cross the rubicon to do it, not anymore anyway.
and also i think its interesting how kids will adjust to this. i think a lot of kids wont hear about this and will find themselves caught up in a child porn case.
im so proud of the responses that people seem to generally have. it makes me feel confident in the future of the world.
isnt there some device to encrypt and decrypt messages with a separate device that couples to your phone? like a device fit into a case and that has a keyboard interface built into a screen protector with indium oxide electrodes.
As far as this is concerned, seems like if you don’t use iMessage or iCloud you’re safe for now.
Yes, this is correct. The Messages feature only applies to children under 18 who are in an iCloud Family, and the photo library feature only applies if you are using iCloud Photos.
1. Send someone you hate a message with cartoon making fun of tyrant-president.
2. That person is now on a list.
Its swatting-as-a-service.
This also means that it is shielded from attack by the power structure. That is the bargain that the tech industry has struck.
The agenda is always towards increasing power for the power structure. One form of power is information. That means that Apple is inexorably drawn towards increasing surveillance. Also, Apple’s massive customer base both domestic and overseas is a juicy surveillance target.
Hint: it isn’t end to end encrypted, Apple doesn’t need your password to read the information, and you will never know
Who the frack would design a system that way and why?
When saying no to ideas like this, we should at the same time attempt to also share our thoughts on what would be an acceptable alternative solution.
Considering they hold the keys and the scheme already allows them to decrypt as a last step the users photos, this is not exactly a progress. It just maintains the illusion that those backups are encrypted while they (ultimately) aren't.
I've personally (and some may disagree) always assumed that anything you put in any cloud (and that includes the very convenient iCloud backups that I use) is fair game for local authorities, whether that's true in practice or not.
Putting a "snitch" on device, even if it's only for content that's going to the cloud (and in the case of an iCloud backup, doesn't that mean all your iPhone content ?) is the part that goes a step too far and will lead to laws in other countries asking for even more.
Once you've opened the door to on device scanning, why limit it to data that goes to iCloud ? Why limit it to photos ? They proved they have the "tech" and governments around the world will ask for it to be bent to their needs.
I'm sure the intent was well meaning but I'd much rather they just do this on their premises and not try to pretend they do this for privacy.
Facebook has been making it harder for random strangers to contact people under a certain age, so that may well help, and we'll see if it does. And we could probably teach teenagers how to remain safe on the internet, and give the support needed to not be too emotionally reliant on the internet. That might get you part of the way.
You could run TV advertisements to raise awareness about how abuse is harmful to try to dissuade people from doing it, but that might make the general public more scared of it (the chances their family specifically will be affected has to be remote), and more inclined to "regulate" their way out of the problem.
You could try to take more children away from their families on the off-chance they may have been abused, but what if you make the wrong call? That could be traumatizing to them.
You could go down the road of artificial child porn to compete with child porn, and robots which look like children, but I don't think the predators specifically are interested in those, are they? And that comes with some serious ethical issues, and is politically impossible.
We can't just profile "whoever looks suspicious" on the street, because people who are mentally ill tend to behave erratically, only have a slightly high chance of being guilty, but have a dramatically high chance of being harassed by police.
If we can get out of the covid pandemic, this may help. Child abuse is said to have risen by a factor of 4 during the lockdowns, and all those other things which were put in place to contain the virus. It's possible that stress from the pandemic, and perhaps, opportunities to commit a crime may have contributed to this. But, this is an international problem, even if the pandemic were to vanish in the U.S., it may still exist overseas.
1. https://puri.sm/products/librem-5/
2. https://puri.sm/posts/librem-5-update-shipping-estimates-and...
If I had to guess, I'd hunt around for a Windows tablet that someone had good luck running Linux on. Maybe a Surface Pro.
“ When Apple releases these “client-side scanning” functionalities, users of iCloud Photos, child users of iMessage, and anyone who talks to a minor through iMessage will have to carefully consider their privacy and security priorities in light of the changes, and possibly be unable to safely use what until this development is one of the preeminent encrypted messengers.”
People sending messages to minors that trigger a hash match have more fundamental things to consider, as they are sending known photos of child exploitation to a minor.
The EFF writer knows this, as they describe the feature in the article. They should be ashamed of publishing this crap.
Additionally, you are not “obliged” to report such photos to the police. Uninvolved service providers do have to submit some sort of report iirc, but to require regular users to do so would raise Fifth Amendment concerns.
How do you know its a known photo of child exploitation? The original image that was hashed and then deleted. Two completely different images have the same hash.
WhatsApp automatically saves images to photos. What if you receive a bad image and are reported due to someone else sending the image to you?
Think of it this way: If you want to hide from companies, choose Apple. If you want to hide from the US Government, choose open source.
But if your threat model really does include the US government or some other similarly capable adversary, you are well and truly fucked already. The state-level apparatus for spying on folks through metadata and traffic interception is now mode than a decade old.
It's not just the US government: they've been cooperating with the PRC government as well (e.g. iCloud in China runs on servers owned by a state-owned company, and apparently China rejected the HSM Apple was using elsewhere, so they designed one specifically for China). Apple has some deniability there, but I personally wouldn't be surprised if China could get any data from them that it wanted.
https://www.nytimes.com/2021/05/17/technology/apple-china-ce...
The best hopes against a population-wide Chinese-style social credit system being implemented in the US remain constitutional and cultural, but the more architectural help we get from technology the better. “Code is law” is still a valid observation.
Expanded Protections for Children - https://news.ycombinator.com/item?id=28078115 - Aug 2021 (291 comments)
Apple plans to scan US iPhones for child abuse imagery - https://news.ycombinator.com/item?id=28075021 - Aug 2021 (349 comments)
Apple enabling client-side CSAM scanning on iPhone tomorrow - https://news.ycombinator.com/item?id=28068741 - Aug 2021 (680 comments)
“ Compromising the security of our personal information can ultimately put our personal safety at risk. That is why encryption has become so important to all of us.”
“… We have even put that data out of our own reach, because we believe the contents of your iPhone are none of our business.”
“ The FBI may use different words to describe this tool, but make no mistake: Building a version of iOS that bypasses security in this way would undeniably create a backdoor. And while the government may argue that its use would be limited to this case, there is no way to guarantee such control.”
Tim Cook, 2016
Can they trust random government to give them a database of only CSAM hashes and not insert some extra politically motivated content that they deem illegal ?
Because once you've launched this feature in the "land of the free", other countries will require for their own needs their own implementation and demand (through local legislation which Apple will need to abide to) to control said database.
And how long until they also scan browser history for the same purpose ? Why stop at pictures ? This is opening a very dangerous door that many here will be uncomfortable with.
Scanning on their premises (considering they can as far as we know ?) would be a much better choice, this is everything but (as the "paper" linked tries to say) privacy forward.
“The scheme will initially roll out only in the US. […] Apple’s neuralMatch algorithm will continuously scan photos that are stored on a US user’s iPhone and have also been uploaded to its iCloud back-up system.”
Researchers interviewed for the article would agree with your analysis. “Security researchers [note: appears to be the named security professors quoted later in the article], while supportive of efforts to combat child abuse, are concerned that Apple risks enabling governments around the world to seek access to their citizens’ personal data, potentially far beyond its original intent.”
Article link for ease of access: https://www.ft.com/content/14440f81-d405-452f-97e2-a81458f54...
https://www.vox.com/recode/2021/5/13/22435266/apple-employee...
https://www.vox.com/recode/22583549/apple-employees-petition...
Will they apply that energy and leverage to push back on this?
How else can this be stopped before it goes too far? Telling people to "Drop Apple" is even less effective than "Delete Facebook".
This kind of stuff absolutely petrifies me because I’m so scared of getting accidentally scooped up for something completely unintentional. And I do not trust police one bit to behave like intelligent adult humans.
Right now I feel like I need to stop doing ANYTHING that goes anywhere outside the velvet ropes of the modern commercial internet. That is, anywhere that cannot pay to moderate everything well enough that I don’t run the risk of having my entire life ruined because some #%^*ing algorithm picks up on some content I didn’t even choose to download.
No, only if you save multiple CSAM images to your photo library and have iCloud Photo Library turned on.
NSO used exploits in iMessage to enable them to grab photos, texts among other things.
Now shortly after Apple security patches we see them pivot and now want to “work” with law enforcement. Hmmm almost like once access was closed Apple needs a way to justify “opening” access to devices.
Yes I realize this could be a stretch based on the info. Just seems like an interesting coincidence… back door exposed and closed…. now it’s back open… almost like governments demand access
But now I'm reminded of how fucking awful and hostile Apple and other companies can be. I'm once again 100% convinced that free software is the only way to go, even if I have to endure using software with ugly UIs and bad UX. It will be worth it just not to have to use software written by these assholes.
Stallman was right.
https://techcrunch.com/2021/04/28/apple-record-china-2021/
Apple's iPhone revenu just doubled from last year in China -- now 17 billion. Thats not a small number. The play against Huawei has done it's job, apparently -- it's quite mortally injured.
For sure the CCP would love to scan everyone's phones for files or images it finds troubling and for sure every country will eventually be allowed to have its own entries in this database or even their own custom DB.
So my cynical side says...Apple just sold out. MASSIVELY. The loosers -- everyone pretty much that buys their phones.
But then I have to remind myself, the old Apple is long gone, the new Apple is a completely different beast, with a very different concept of what it is marketing.
Teens are not stupid. They'll eventually clue-in that big brother is watching and won't appreciate it. They'll start by using other messengers instead of imessage and then eventually leaving the ecosystem for Android or whatever else comes down the pike in the future.
I'd like to get verification but that hopefully means your scenario is unlikely.
> "Apple is planning to build a backdoor into its data storage system and its messaging system"
The only practical barrier here is that their parents have educated them and their mental model arrives by its own at "no, this is a very bad idea" instead of "yes, I want to send this pic". Anything else, including petty prohibitions from their parents, will not be a decision factor in most cases. Have we forgotten how it was to be a teenager?
(I mean people, both underage and criminals, will just learn to avoid apple and use other channels)
If was a conspiracy type, I would assume this is more likely to be apple responding to an NSA request to de-crypt data.
This idea will be gradually expanded:
1. To detect child abuse (unsuccessfully)
2. To detect terrorism (also unsuccessfully)
3. To detect criminal activity (successful only against low-level criminals)
4. To detect radical political views as defined by Apple corporation
5. To detect human behaviors that are not supported by Apple's corporate vision
No, they aren't, categorically. That's why they keep getting caught that way.
Apparently I was wrong, I loved apple products and ecosystem. Not sure what to switch after this :/
Having a hard time buying this is about "the kids" or children in any way, shape or form. This is typical erosion of privacy under a worn out flag, just more emotional manipulation.
Have you seen what smartphones have done to people, especially children? Apple, Google, Facebook, Twitter, the whole lot of them. They are out to destroy children, not save them. If they thought they could "generate" 1 more dollar in "value" they'd be selling these abhorrent images to the highest bidder.
It seems this can then be a security risk, since Apple could be breached and they'd have the means to server side decrypt things.
If it was simply that client side end to end encryption can be turned on/off based on if the account is a child account or not (or as a configuration for parental control) that be different.
As just a config, then I mean the slippery slope always existed, Apple could always just be forced into changing the settings of what gets end to end encrypted and when.
But if this means that all photos are sent unencrypted to Apple at some point, or sent to Apple in a way they can decrypt, then it does open the door to your photos not being securely stored and attackers being able to steal them. That seems a bit of an issue.
They have the encryption key that allows them to read their customer data.
_Hashes_ of photos will be scanned for _known_ abusive material, client side.
So the only thing Apple can find out about you is if you have some of these known and catalogued images. They will definitely not know if you have other nude photos, including of your children.
The other, separate feature is a parental control feature. You as a parent can be told if your children send or receive nude photos. This obviously sacrifices some of their privacy, but that is what parenting is. It's not more intrusive than screentime, or any number of things you might do as a parent to make sure your children are safe..
Does anybody have recommendations on what to do to help oppose this instead of just feeling helpless?
This is the downside to upgrading your iOS version. Once you update, it's not like you can go back, either. You're stuck with a slower, more power-hungry phone for the life of the phone.
Is there anyone who's familiar with the technology so they can explain how it works?
Probably using some sort of probabilistic query like a bloom filter.
“ It is used on Microsoft's own services including Bing and OneDrive,[4] as well as by Google's Gmail, Twitter,[5] Facebook,[6] Adobe Systems,[7] Reddit,[8] Discord[9] and the NCMEC,[10] to whom Microsoft donated the technology.”
Any kind of machine-based contextual analysis of users' content will be a disaster.
searching for CP is the original pretext
Convince me that a strong step to ending CSA at the expense of a little privacy is a bad thing.
This technology uses secret sharing to ensure a threshold of images are met before photos are flagged. In this case, it's even more private than CCTV.
Totalitarian regimes to do not need some magic bit of technology to abuse citizens; that's been clear since the dawn of time. Those who are concerned about abuse would do well to direct their efforts towards maintenance of democratic systems: upholding societal, political, regulatory and legal checks and balances.
Criminals are becoming better criminals by taking advantage of advancements in technology right now, and, for better or worse, it's an arms race and society will simply not accept criminals gaining the upper hand.
If not proven necessary, society is capable of reverting to prior standards (Habeas Corpus resumed after the Civil War, and parts of the Patriot Act have expired, for example.).
- User: Are you out of your f... mind?
- Apple: It's for children protection.
- User: Ah, ok, no problem, please install spyware and do later whatever you wish and forget about any privacy, the very basis of rights, freedom and democracy.
This is by the way how Russia started to filter the web from political opponents. All necessary controls were put in place under the same slogan: "to protect children"
Yeah, right.
Are modern people that naive and dumb and can't think 2 steps forward? Is that's why it's happening?
Edit: Those people would still need to explain how living in society without privacy, freedom and democracy with authoritarian practices when those children will grow up will make them any 'safer' ...
> Apple’s method of detecting known CSAM is designed with user privacy in mind. Instead of scanning images in the cloud, the system performs on-device matching [...]
It's incredible that Apple arrived at the conclusion that client-side scanning that you cannot prevent is more private than cloud-scanning.
Since they claim they're only scanning iCloud content, why not scan in the cloud?
They decided the most private way is to scan iCloud content before it's uploaded to the cloud... Because if they scanned in the cloud it would be seen as a breach of privacy and is bad optics for a privacy-focused company? But scanning on the physical device that they have described as "personal" and "intimate" has better optics? That's amazing.
This decision can only be read as Apple paving the way to scanning all content on the device, to bypass the pesky "Backup to iCloud" options being turned off.
A more recent example is how private set intersection became an easy way to get contact tracing tech everywhere while maintaining an often perfunctory notion of privacy.
I wonder where large companies will take this next. It behooves us cryptography/security people who actually care about not walking down this slippery slope to fight back with tech of our own.
This whole thing also somewhat parallels the previous uses of better symmetric encryption and enclaves technologies for DRM and copyright protection.
Chilling. Why have human reviewers, unless false positives are bound to happen (this is of 100% certainty with the aggregate amount of photos to be scanned)?
So, in effect, Apple has hired human reviewers to police your photos that an algorithm has flagged. Whether you knowingly consent to or not (through some fine print), you are being subjected to a search without probable cause.
This is not the future I was looking forward to.
(2) I'm equally intrigued by the paradox that in order for the algorithms that perform the CSAM detection to work, it must require some data set that represents these reprehensible images (which are illegal to possess).
Not that you care, but this is the straw that's broken this camel's back. It's too ripe for abuse, it's too invasive, and I don't want it.
You've used one of the Four Horsemen of the Infocalypse perfectly… and so I'm perfectly happy to leave your ecosystem.
Cheers.
Well, yes? Parents are already legally responsible for their young children and under their supervision. The alternative would be to not even give such young children these kind of devices to begin with - which might actually be preferable.
> this system will give parents who do not have the best interests of their children in mind one more way to monitor and control them
True. But the ability to send or receive explicit images would most likely not be the biggest issue they would be facing.
I understand the slippery slope argument the EFF is making, but they should keep to the government angle. Having the ability for governments to deploy specific machine learning classifiers is not a good thing.
What are some other fully encrypted photo options out there?
Then this news broke. Apple, you just lost several thousand dollars in sales from me. I had items in cart and was pricing everything out when I found this news. I will spend my money elsewhere. This is a horrendous blunder. I will not volunteer myself up to police states by using your gear now or ever again in the future. I've even inquired about returning the work laptop in exchange for a Dell.
Unsafe at any speed. Stallman was right. etc etc etc.
1. They expect must people will shrug and let themselves be scanned. That is, this privacy invasion will result in minimal damage to the Apple brand, or
2. They know privacy-savvy people will put them from now on on the same league with Android, and they are prepared to take the financial loss.
Scenario 1 is the most plausible, though it hints an impish lack of consideration for their customers.
Scenario 2 worries me most. No smart company does something counter-productive financially unless under dire pressure. What could possibly make Apple shoot itself on the foot and announce it publicly? In other words, Apple's actions, from my perspective, look like a dead canary.
This would be super useful for iPhone security, e.g. incoming files could be scanned for attempting to use (closed) exploits, when the user can easily associate a malicious media file with the message sender or origin app/site.
On jailbroken devices (e.g. iPhone 7 and earlier with unpatchable boot ROMs), is there a Metasploit equivalent for iOS, which aggregates PoCs for public exploits?
A related question: will PhotoDNA hashing take place continuously or in batch, e.g. overnight? How will associated Battery/Power usage be accounted, e.g. attributed to generic "System" components or itemized separately? If the former, does that create class-action legal exposure for a post-sale change in device "fitness for purpose"?
And I bet that Saudis and other oppressive regimes will use this to detect other “crimes”.
So I'll propose an alternative theory: Apple is doing this not to actually catch any child pornographers, but to ensure that any CP won't actually reach their servers. Less public good, more self-serving.
Good question. Companies have to follow laws. The naive, early 2000s notion that the internet was unstoppable and ungovernable was mistaken. Apple, Google and the other internet bottlenecks were, it turned out, the pathway to a governable internet. That fight is lost.
Now that it's governable, attention needs to be on those governing... governments, parliaments, etc.
The old version of freedom of speech and such didn't come from the divine. They were created and codified and now we have them. We need to do that again. Declare new, big, hairy freedoms that come with a cost that we have agreed to pay.
There are dichotomies here, and if we deal with them one droplet at a time, they'll be compromised away. "Keep your private messages private" and "Prevent child pornography and terrorism in private messages" are incompatible. But, no one is going to admit that they are choosing between them... not unless there's an absolut-ish principle to defer to.
Once you're scanning email for ad targeting, it's hard to justify not scanning it for child abuse.
In place of Mail: Tutanota In place of iMessage: Signal And so on…
Question - if most people literally don't want to have anything to do with CP, isn't uploading of a hash database of that material to their phones precisely that?
For once I think I will feel disgusted walking around with my phone in a pocket; a phone that is full of hashes of child porn. That's a terrible feeling.
“The threshold is set to provide an extremely high level of accuracy and ensures less than a one in one trillion chance per year of incorrectly flagging a given account.”
How did they calculate this? Also, I can imagine more than a trillion photos being uploaded to iCloud a year.
I'm missing how this will actually work if perpetrators knew Apple was going to analyze their data beforehand. Could someone explain?
Separate from kids, I wonder whether Apple's is yet shooting itself in the foot for teens.
Teens should start caring about privacy around then, are very peer/fashion-sensitive, and have shown that they'll readily abandon platforms. Many parents/teachers/others still want to be treating teens as children under their power, but teens have significant OPSEC motivation and ability.
Personally, I'd love to see genuinely good privacy&security products rushing to serve the huge market of a newly clueful late-teen generation. The cluefulness seems like it would be good for society, and market forces mean the rest of us then might also be able to buy products that aren't ridiculously insecure and invasive.
I understand that it doesn't scan everything, but it don't matter. What matter is there's an implemented technical capability to run scans against external fingerprint database. it's a tool which may be used for many needs.
I hope some countries will prohibit Apple doing that. Germany with its strict anti-snooping laws comes to mind. Maybe Japan. The more, the better.
Oh, and by the way, every tech-savvy sex predator now knows what they should avoid doing. As always with mass privacy invasions: criminals are the last to suffer from it.
Yet....dumb enough to upload it unencrypted to iCloud instead of storing it in a strongly encrypted folder on their PC?
The two circles in this diagram have a very thin overlap I think.
Dumb move by Apple, privacy is either 100% private or not private.
Unless somebody can enlighten me that like 23% of all investigated pedophiles that had an iPhone seized had unencrypted CP on their iCloud accounts? I am willing to be proven wrong here.
I'm looking into privacy phones for the first time and will be switching.
>"The (unauditable) database of processed CSAM images will be distributed in the operating system (OS), the processed images transformed so that users cannot see what the image is, and matching done on those transformed images using private set intersection where the device will not know whether a match has been found"
Am I reading this correctly in that Apple will essentially be pushing out contraband images to user's phones? Couldn't the existence of these images on a user's phone potentially have consequences and potentially be used against an unwitting iPhone user?
Apple, very astutely, understands that difference and exploited the latter to differentiate its phones from its main competitor: cheap(er) android phones.
Apple didn’t want the phones to be commoditized, like personal computers before it. And “privacy” is something that you can’t commoditize. Once you own that association, it is hard to fight against it.
Apple also understands that the general public will support its anti child exploitation and the public will not see this as a violation of privacy.
* knock knock * "we received an anonymous report that you have hate speech an illegal meme on your phone, please come with us"
The question will be if Apple will bend to requests to leverage this for other reasons less noble than the protection of children. Apple has a lot of power to say no right now, but they might not always have that power in the future.
It just seem very paradoxical to be using a cloud based photo and/or un-encrypted backup service and then worry about one’s privacy being at risk.
You just have to say for the greater good and you can get away with anything. Over the last year and half so many have been desensitised to over bearing collectivism that at this stage i think governments and their any Big Corp lackeys could get away with just about anything now.
I expect that any time you take a photo, the scan will be performed right away, and the results file will be waiting to be sent the next time you enable voice and data.
This capability crushes the trustworthiness of the devices.
I mean they use the privacy argument to avoid side-loading apps, lol. But scanning your photos is OK.
What absolute hypocrisy.
If you don't trust the rule of law, Apple can't fix that for you.
For context, see https://news.ycombinator.com/item?id=20620102
Apple has altered the deal. Pray they do not alter it any further.
Now you have to live with the consequences of convenience.
I’d rather not have that on my phone.
I don't use apple products, but if I found out google was scanning my photos on photos.google.com on behalf of the government I would drop them. I'm not saying it wouldn't hurt, because it definitely would, but in a capitalistic country this is the only way to fight back.
As such, it should NEVER do anything that isn't in your best interest-- to the greatest extent possible under the law. Your relationship with your personal computer is closer and more trusted than your relationship with your doctor or lawyer-- in fact, you often communicate with these parties via your computer.
We respect the confidentiality you enjoy with your professional agents but that confidentiality cannot functionally exist if your computing devices are not equally duty bound to act in their users best interest!
This snitching 'feature' is a fairly general purpose tracing/tracking mechanism-- We are to assume that the perceptual hashes are exclusively of unlawful images (though I can't actually find a firm, binding assertion of that!)-- but there is nothing assuring that to us except for blind trust.
Even if the list today exclusively has unlawful images there is no guarantee that tomorrow it won't have something different-- no guarantee that some hysterical political expediency won't put images associated with your (non-)religion or ethnicity into it, no guarantee that the facility serving these lists won't be hacked or abused by insiders. Considering that possession of child porn is a strict liability crime, Apple themselves has presumably not validated the content of the list themselves and certainly you won't be allowed to check it. Moreover, even if there were some independent vetting of the list content there is nothing that would prevent targeted parties from being given a different unvetted list without their knowledge.
The pervasive scanning can also be expected to dramatically increases the effectiveness of framing. It's kind of cliche that the guilty person often claims "I was framed"-- but part of the reason that framing is rare is because the false evidence has to intersect a credibly motivated investigation, and they seldom do except where there are other indicators of guilt. With automated scanning it would be much more reliable to cause someone a world of trouble by slipping some indicated material on their device, and so framing would have a much better cost/benefit trade-off.
Any of the above flaws are sufficiently fatal on their own-- but add to it the potential for inadvertent false positives both in the hash matching and in the construction of the lists. Worse, it'll probably be argued that the detailed operation of the system must be kept secret from the very users whos systems it runs on specifically because knowledge of the operation would greatly simplify the malicious construction of intentional false positives which could be used for harassment by causing spurious investigations.
In my view Apple's actions here aren't just inappropriate, they're unambiguously unethical and in a more thoughtful world they'd be a violation of the law.
[1] https://www.vox.com/the-goods/2019/6/4/18652228/apple-sign-i...
> OS and iPadOS will use new applications of cryptography to help limit the spread of CSAM online, while designing for user privacy. CSAM detection will help Apple provide valuable information to law enforcement on collections of CSAM in iCloud Photos.
WhatsApp is not a hosting service.
So where's the news?
