>the system is just a few bit flips away from scanning every photo on your device
I prefer to think of it as being just one national security letter away from that happening.
Which is of course a schroedinger's cat kinda thing. It's already been sent. Or it hasn't. But why would the national security apparatus not take advantage of this obvious opportunity?
Anyone giving benefit of the doubt to this kind of stuff nowadays is IMO very naive or just poorly informed of recent digital history.
How would the agency issuing an NSL be able to generate a hash of a photo they’re looking for? Presumably if they already had a photo to derive the hash they’d already have whatever it is they’re searching for.
Because you're searching for people, not documents.
For a normal law enforcement context, say you bust 1 drug dealer, and are trying to find more. Maybe they have an image of a "contract" (document outlining terms of a drug deal) on their phone. You could find the other parties to that contract by searching everyone's phones for that image.
For a national security context, you could imagine equivalents to the above, you could also imagine searching for classified documents that are believed to have been leaked, or maybe searching for documents you stole from an adversary that you believe there spies are likely to be storing on their phones.
I'm saying documents here instead of images, but plenty of documents are just images, and I have little doubt that they could get this program to expand to include "scan PDFs, because you could make a CSAM into a pdf" (if it doesn't already?).
I think you already got one great reply, I have just one thing to add to it: your post literally presupposes a 1.0 version of this software that never has its feature set expanded. I don't think that's a reasonable assumption. After all, with 1.0 the goal of catching this class of person is barely achieved. They'll likely arrest people, 99% of whom are just CSAM jpeg collectors who get an extra kick out of viewing a class of image that is super illegal. And nothing more.
Then for version 2.0, they'll realize the INTERPOL, FBI, whatever can provide them a combo hash plus one string of text that can nail a known active producer. The holy grail. This small feature add to get so much closer to the original "moral goal" will prove too appealing to pass up. Now all the code is in place for the govt to pass over requests for data pinpointing mere common criminals.
The program has capability to upload questionable photos for review.
Just make it do equivalent of .* of all photos on device. It would be hard to argue, that's more difficult than scanning for specific hash.
And there is nothing specific about images. Extending this to scan arbitrary data is probably not that much code, depending on how its program maybe configurable.
I prefer to think of it as being just one national security letter away from that happening.
Which is of course a schroedinger's cat kinda thing. It's already been sent. Or it hasn't. But why would the national security apparatus not take advantage of this obvious opportunity?
Anyone giving benefit of the doubt to this kind of stuff nowadays is IMO very naive or just poorly informed of recent digital history.