> To do that right would be a full time project for a handful of people for at least months, not "a one line powershell command"
Huh? Maybe bash and not powershell, but it should be pretty simple. If they have to go into the tape backups for that data, ok, perhaps not, but a reasonable mail system should make this a fairly simple task. Even if it takes a 30-line python script, that's not a big deal, and it can be left to run unattended over how many hours or days it needs to in order to go through that many files.
> You then harangued them for charging too much -- they had to come up with an estimate, and they estimated 30 seconds to review the data they sent you for each email.
The city later admitted that its original estimate was completely wrong, because they didn't have to review the email data at all (since bodies weren't to be included). I think "harranguing" them (which there's no evidence the OP did) seems pretty reasonable when they wanted to charge tens of millions of dollars to do something that ultimately actually cost forty bucks.
Also understand that this is in general how FOIA is: people ask for information, and government agencies do whatever they can to get out of providing that information, regardless of their legal requirement to do so. It's entirely reasonable to look at initial objections from the agency with a cynical, skeptical eye.
> You then insisted they __agree not to review the data__ so you could get it cheap
Huh? The data did not legally require review. The city made a mistake initially, or intentionally lied in order to get out of fulfilling the request.
> You should have informed them, deleted the data, and moved on with your life.
That's... exactly what he did? His initial beating-around-the-bush response was designed with the hard-earned knowledge that people often get punished for reporting mistakes of this nature. Which... is what actually seemed to be happening after the city was actually aware of their self-inflicted data breach. They wanted him to submit to a third-party audit (no, screw that) and seemed to be moving in the direction of filing charges against him before he got his lawyer involved. For their own negligence!
I have zero sympathy for the city here. Regardless of why OP wanted this data or whether or not you or I think it's useful data for a citizen to have, the city screwed the pooch here, and OP's karma slate seems pretty clean to me, at least for this particular incident.
Should it be that easy to socially engineer city employees with access to huge troves of potentially confidential information?
If anything, OP did the city a favor by going out of his way to inform them. What if OP was a black hat?
Also, who cares why he wanted the data? It's his right to it as a citizen. Just like it's your right to drive around at 2am without answering to the cops, even if they find it weird.
