Hacker News new | past | comments | ask | show | jobs | submit login

Yeah, of course the collision rate in an adversarial dataset is likely to be much higher.

But I really wonder why you think this is an important objection, do you think a lot of people want to go to the "get flagged for child porn" casino?




Themselves? No. Other people and communities that they dislike? You bet.

Just look at political memes. What is the chance that both sides will have a few people try to create memes for their opponents which are adversarial. They'll take care to make sure those images aren't on their own apple devices before spreading it to areas where the other side likes to share memes.

Another example are the very people trying to be caught. Someone who is against current laws about the subject might seek to create as many false positives as possible to overwhelm the system. They might even specifically target otherwise legal baby photos to manipulate in this way as it would make it more likely they get past any sort of second tier manual review and result in law enforcement wasting resources.

Lastly, this is nothing new. Planting such material and flooding websites with it has long been a tactic used by some. Up until now it has been limited because doing so requires violating the law yourself and few people hate others enough to go through that level risk to self. But this is mostly risk free because creating adversarial images like this isn't outright illegal and even in cases where there are laws against it violating those laws is extremely different than violating actual laws against CSAM in every way that factors into a person's willingness to break laws.


The existence of a preimage attack makes Apple's system completely useless for its nominal purpose. The NeuralHash collider allows the producers and distributors of CSAM material to ensure that nearly all of the next generation of CSAM will suffer from hash collisions with perfectly innocent images.

If these new images never make it to the NCMEC database, then new CSAM content will be completely NeuralHash-proof. However, if these images eventually make their way into the NCMEC database, then everybody who has the perfectly innocent originals will be dealing with an adversarial environment.


> If these new images never make it to the NCMEC database

The remark that an image might not be in the database applies equally to all CSAM pictures, and so is not actually dependent on any technical aspect of the hash at all.

The fact that you can modify an image to the point that it gets a different hash isn't important compared to the (separate) issue of how you keep your database up to date. And detecting old CSAM is in any case not as important as tracking and interdicting production of new images.

The second remark is good, but mainly because it reminds us to demand Apple ensures that additions are scanned for malicious images before being added to the central database. If the operator determines that a certain images collides with some known public image they can notify Apple. Apple in turn modifies the hasher to dissolve the collision, roll out a patch, and can then update the database.


> ensure that nearly all of the next generation of CSAM will suffer from hash collisions with perfectly innocent images.

Even then, for that to affect _John Doe_, they would have to make 30+ images whose hash matches that of images in _John Doe’s_ iCloud account.

I think that means they could target individuals, but only if they knew or could guess what photos they have in their account.

They also might be able to target groups of individuals, say people who went on holiday to Paris. It would be interesting to see whether such people have enough overlap in the sets of Neuralhashes of photos they took there.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: