The proposed attack on Apple's protocol doesn't work. The user's device adds randomness when generating an outer encryption key for the voucher. Even if an adversary obtains both the hash set and the blinding key, they're just in the same position as Apple—only able to decrypt if there's a hash match. The paper could do a better job explaining how the ECC blinding scheme works.
This is one of the concerns in the OP, have an AI generate millions of variations of a certain kind of images and check the hashes. In this case it boils down to how common false positives neural hashes are.
> The proposed attack on Apple's protocol doesn't work.
With all due respect, I think you may have misunderstood the proposed attack @jonathanmayer, as what @jobigoud said is correct.
Given some CP image, an attacker could perhaps morph it into an innocent looking image while maintaining the hash. Then spread this image on the web, and incriminate everybody.
Here is a proof of concept I just created on how to proceed : https://news.ycombinator.com/item?id=28105849
Of course I'd dare not research or tinker with it lest I'll be added to a list somewhere such is the chilling effect.
I guess in that case they'd delete that single hash from the database because they'd still have an endless (sadly) supply of other bad image hashes to use instead.
You'd still have to generate several images and persuade people to download multiple of them into their photo roll. And as I understand it there's yet another layer of Apple employees to review the photo metadata before it ever makes its way to law enforcement.
The focus on CSAM seems extremely hypocritical when authorities make such little effort to stop ongoing CSA. I would encourage everyone to research the Sophie Long case. Unless there is image or video evidence the police make little effort to investigate CSA because it's resource intensive.
But PhotoDNA has been scanning cloud photos (Google, Dropbox, Microsoft, etc.,) to detect CSAM content for a decade now and this "pretty soon it's terrorism" slippery slope hasn't yet manifested, has it?
If the slope was going to be slippery, wouldn't we have seen some evidence of that by now?
In case you didn't the topic, what is specific (for now, for now...)to iCloud/apple is the "we're scanning your photos on your device and maybe reporting them if they're bad" approach. So you get the local hashes on the supposedly encrypted files and you get the situation of local files trigger global effects like the police swooping down and arresting you. So that's why despicable and hair-brained scheme in specific produces a greater "attack surface" in multiple ways.
And again, sure, Apple doing this quite possibly will set a precedent for Google et al to answer the other ambiguous meanings your ambiguous comment has.
Right now they are able to bill this as doing what they currently do server side, but client side. Later, they can say they are simply applying the same "protections" to all photos instead of merely the ones being uploaded to iCloud.
For example, we already now have a tool for generating NeuralHash hashes for arbitrary images, thanks to KhaosT:
Only if they're being sent to or from a minor, I thought?
But I haven't looked to closely into it.
They have a system that checks for hashes of images to try and find specific CSAM from a database when images are uploaded to iCloud, this already happens but is now moving on device. When explaining this I've used the analogy that here they are looking for specific images of a cat, not all images that may contain a cat. When multiple images are detected (some threshold not defined) it triggers an internal check at apple of details about this hash and may then involve law enforcement.
The other one is for children 12 and under, that are inside a family group. The parents are able to set it up to show a pop up when it detects adult content. In this case they are looking for cats in any image, rather than specific cat image. The popup lets them know it may be an image not suitable for kids, that its not their fault and they can choose to ignore it. It also lets them know if they chose to open it anyway their parents will get notified and be able to see what they've seen.
This is a good rundown: https://www.apple.com/child-safety/
In this case, the question assumes that an attacker would more or less be creating their own database of hashes and derived keys (to search for and decrypt known photos and associate them with user accounts, or to bruteforce unknown photos), and would therefore have no need to worry about acquiring the key used for blinding the CSAM hash database.
Decrypting vouchers requires the server blinding key and the NeuralHash derived metadata of the input image (technical summary page 10, Bellare Fig. 1 line 18). This attacker only has the latter.
But think of the children and security of the society. Couple that with constant monitoring of your car and you can be monitored anywhere
Maybe they can also build an api so that governments can search easily for dissidents without the delays that the due process of law causes.
The best of both worlds: keep advertising their privacy chops to the masses, while also allowing any and every government agency a programmatic way to hash-verify the data passing through their systems in real-time.
