For what it’s worth, when I worked there, all the engineers I met sincerely cared about just…making a useful product, and respecting people’s data in the process. Pretty much the only guaranteed fireable offense was looking at someone’s data without permission and a valid reason, e.g., to directly fix something broken about their account, which almost never required viewing anything private anyway.
Nobody appeared to be “casually misusing” data—I think the problem is that they’re largely just engineers, particularly young ones, naïvely considering only the engineering side of things. All the data queries go through the robust privacy-checking system, so everything is good, right?
In a case like this, they didn’t consider the optics of what happens when someone scrapes the public (at the time) profiles of Facebook users and uses that information for nefarious deeds. What happens when users are angry not because their private data was “breached”—a technical problem with an engineering solution—but because they didn’t realise how much they’d already shared publicly (even if you explicitly told them) and how it could be used to influence them en masse?
One of the problems with the Facebook API is that it is disconnected from policy on too many points. The policy is all hand-wavy honor system, and the API lets you trample all over the policy.
Case in point, one of the most common policy violations is prefilling the user message on posts made via the API. It is forbidden. But the field is right there for you to abuse and put whatever you want into it. Sure there are some automated enforcement algorithms and policy employees look at things when complaints go up, but if the policy says you can't do it, why on earth does the code allow it?
OK I know the pat answer is that apps are allowed to prompt the user earlier in the workflow for the message, and then use that value when calling the API. That is true but weak (what would it hurt to eliminate that loophole vs. the benefit of no longer having to detect and take enforcement action on an impossible action) -- the point remains, if they really cared about their vaunted policy and protecting the user, they would put more controls directly into the code behind the API to disallow prohibited actions.
These are things where smart engineers can make a difference. Spend some time on the FB Developer Community Group and you will see the flood of questions from developers who are completely ignorant of the policy, even on basic things like "don't use an account with a name other than your own" aka, there are no business or developer accounts. Many of them willfully ignore policy and just do what the code allows them to do. A lot of good could be done by FB devs taking more accountability for how the platform is abused.
It is not so much that something is wrong, but that everything is working as it should. The system is the problem.
Case in point, Cambridge Analytica used ill-gotten data from 50 million people to craft extremely effective political ads. And since user engagement with those ads was very high… Facebook's algorithm made it cheaper for them to buy even more ads.
Nobody appeared to be “casually misusing” data—I think the problem is that they’re largely just engineers, particularly young ones, naïvely considering only the engineering side of things. All the data queries go through the robust privacy-checking system, so everything is good, right?
In a case like this, they didn’t consider the optics of what happens when someone scrapes the public (at the time) profiles of Facebook users and uses that information for nefarious deeds. What happens when users are angry not because their private data was “breached”—a technical problem with an engineering solution—but because they didn’t realise how much they’d already shared publicly (even if you explicitly told them) and how it could be used to influence them en masse?