What's actually happening is that a mechanism is being introduced that makes it easy to censor anything Apple wants censored. Apple promises to only use it for child pornography.
Apple, however, is still not a sovereign state, and as such must bow to the wishes of actual sovereign states. Sovereign states have proven again and again that they will grab as much power as they can, and doing it insidiously, in the form of a private database of hashes of undesired content, is especially attractive to them.
This is not a hypothetical. For a real-world example look to England where its nation-wide internet blocking system is already used beyond its original scope. Or think of what countries like China will certainly do with such a mechanism. Scope creep, in the form of power grabs by nation states, is a realistic concern, based on vast historical experience, not a fallacy.
Apart from the concern of scope creep, there is also the concern of false positives. When deployed on such a scale, there will undoubtedly be perfectly legimate images being flagged. I'm not happy about my phone containing software that's always vigilant, ready to ruin my life over a false positive.
Apple, however, is still not a sovereign state, and as such must bow to the wishes of actual sovereign states. Sovereign states have proven again and again that they will grab as much power as they can, and doing it insidiously, in the form of a private database of hashes of undesired content, is especially attractive to them.
This is not a hypothetical. For a real-world example look to England where its nation-wide internet blocking system is already used beyond its original scope. Or think of what countries like China will certainly do with such a mechanism. Scope creep, in the form of power grabs by nation states, is a realistic concern, based on vast historical experience, not a fallacy.
Apart from the concern of scope creep, there is also the concern of false positives. When deployed on such a scale, there will undoubtedly be perfectly legimate images being flagged. I'm not happy about my phone containing software that's always vigilant, ready to ruin my life over a false positive.