As soon as it is used in a negative way beyond CSAM scanning, it will cause people to sell their phones and stop using apple products.
If Apple starts using the tech to scan for religious material, there will be significant market and legal backlash. I think the fact that CSAM scanning will stop if they push it too far will keep them in check to only do CSAM scanning.
Everyone can agree on using the tech for CSAM, but beyond that I don’t see Apple doing it. The tech community is reacting as if they already have.
Problem one is Apple doesn't know what they are scanning for.
This is by design and actually a good thing.
It becomes a problem because problem number 2:
No one is accountable if someone gets their life ruined over a mistake in this database.
I'd actually be somewhat less hostile to this idea if there was more regulatory oversight:
- laws that punishes police/officials if innocent people are harmed in any way
- mandatory technical audits as well as verification that for what it is used for: Apple keeps logs of all signatures that "matched"/triggered as well as raw files, these are provided to the court as part of any case that comes up. This way we could hopefully prevent most fishing expeditions - both wide and personalized ones - and also avoid any follow up parallel reconstructions.
I'm not saying I'd necessarily be OK with it but at that point there would be something to discuss.
It may be worth taking very seriously that you might be overestimating both how quickly regular people become aware of such events and how emphatically people will react.
If Apple starts using the tech to scan for religious material, there will be significant market and legal backlash. I think the fact that CSAM scanning will stop if they push it too far will keep them in check to only do CSAM scanning.
Everyone can agree on using the tech for CSAM, but beyond that I don’t see Apple doing it. The tech community is reacting as if they already have.