Great story and a great read! I googled the characters afterwards, and there are some interesting addendums & updates:
Looks like Ahmad Abouammo (Twitter’s former head of Middle East partnerships) was arrested in Seattle in Nov 2019, but Ali Alzabarah's escape to Saudi Arabia was successful (at least in terms of being arrested by the US government):
Meanwhile, if you create a new Twitter account today from a VPN and follow 30 people, it will lock you out until you verify a non-VoIP phone number.
Removing the number instantly re-locks the account.
It’s really immoral that they demand identity-linked PII while running such a loose ship, where anyone with enough money can buy their way in to obtain that PII, track you down, and maybe cut you up with a bone saw.
Twitter is complicit in this abuse, considering their explicit technical steps taken to ensure that you cannot use Twitter without exposing yourself to these sorts of criminals in the governments of foreign countries, as well as similar ones in the government of Twitter’s own jurisdiction.
> And while Alzabarah’s job entailed maintaining systems to keep Twitter working properly, his position at the company did allow him access to the private information of many users, including their phone numbers, email addresses, and IP addresses. That meant that in some instances, Alzabarah could not only help unmask an anonymous regime critic, but also pinpoint the person’s location.
> where anyone with enough money can buy their way in to obtain that PII, track you down, and maybe cut you up with a bone saw.
TBF, I think that the vast majority of companies out there are vulnerable to this. I’ve worked for 8 tech companies in my career, none of which did anything beyond a basic background check.
Truly mitigating the problem you’re touching on requires a level of vetting and surveillance that you’d typically see applied to intelligence operatives. I think this is similar to how we view infosec generally: those with sufficient resources will be able to penetrate a network, regardless of the design or execution of network security.
This is letting Twitter off the hook. It is not impossible to protect users personal information, even within a company, to a very limited set of people who actually need it, with audits on when and how they are accessing it, and periodic reviews of everyone's access levels. Mature organizations follow specific standards for this kind of stuff. For a company like Twitter, where the privacy of this information literally can mean life or death, it is unforgiveable to not have a better grip on it (cue some non-technical regional bizdev guy having deep access, as per the article).
That’s a really good point, reducing access would help here.
If you’re a nation state with the resources of Saudi Arabia, it still wouldn’t be impossible to bribe or blackmail an employee who has prod access because companies don’t have much of an eye on employees private lives. They would have the resources and theoretically the incentive to really dig into the social engineering/coercion side of things. You’d be amazed how many people that make incredible salaries are in fairly significant amounts of debt. It’s pretty common in western culture, and that’s a prime opportunity for those kinds of operations.
“prod access” should not be a single large group or access boundary.
Access to non-tokenized PII data stores or the small set of systems that require touching untokenized PII should be a very small compartment, with extremely tight min-two-person deployment/introspection controls and minimal change frequency.
Why hasn't Twitter banned Mohammed bin Salman from their website yet? Surely he has violated their terms of use many times at this point. Do Twitter's rules not apply to him because he's insanely rich?
Yep, I once heard a quote: "rules and laws are like spiderwebs. They are sure to catch insects that cause trouble, but larger animals will just pass on through."
This highlights more than ever that whatever customer data your employees have access to, you need to log every single access to it, and have automatic audits- who should be accessing what? What accesses are surprising?
Seems like something one could build a SaaS business around- send them reports that <user> accessed <fields> about <customer ID> on <date>, along with a copy of attributes and roles about each user. Service could offer deep dives, querying, reporting, along with ML or rule-based flagging to say "That seems odd".
If Twitter can't build the infrastructure needed to do that, I can't imagine how few small companies can do it themselves either.
Telcos have this level of monitoring of accounts because employees routinely would abuse this access to find details of exs, family members, friends and celebrities (billing info, call detail records, etc.). The problem was there was no proactive monitoring - it was all reaction based upon complaints that would kick off the investigation. I asked why this wasn't automatic to detect clear abuse and the answer was "do you know how many people we'd have to fire if we went looking for abuse?".
> "do you know how many people we'd have to fire if we went looking for abuse?"
I think they make the mistake of presuming this has to go up to 11 on day 1.
Tell the employees you're going to be proactively auditing. Choose a threshold. Interview and potentially fire those employees going over the threshold. Do this until you're done firing people, then increase the threshold and repeat. You will have to fire people, or threaten to, but employees will get the message that the PII-party is over.
The challenge was that depending on the role, the union was involved. When that came into the picture there was a long, dragged out grievance process where someone would have to violate the policy x amount of times in y months before they'd be terminated. It was not unusual from what I've seen for folks to abuse this for years.
To be honest tho - this is the contract that both the company and union agreed to, so bad on the company for being okay and not making this a more serious infraction. I've talked to some of the union stewards about this and they basically said they wanted this data locked down harder. They said it's too easy to access and super temping and wished the company would put more protection around it. Go figure.
I know for a fact that The Canadian Revenue Agency has this type of system in place and it isn't reactive to a complaint. It's proactive. If you try to pull up, say, Wayne Gretzky's tax information the system is able to detect whether or not you have a likely need for the information. If not, you're temporarily denied access until your access is evaluated by humans that are capable of asking why you need access. "For funsies" is not an acceptable response.
If they're abusing their position they don't deserve to keep their job. The companies complicity makes them party to whatever these staff do. Whether a lawyer can prove that or not depends on the depth of the aggrieveds bank account.
The same problem existed in patient data prior to HIPAA. It still happens, but the unauthorized disclosures have gone down due to automated trips. It used to be your check in clerk could run any reports on whomever they wanted without giving it a second thought.
So when I worked for the cable company straight out of uni and we would look up famous local people to see what packages they subscribed to, we should have all been fired? I guess technically, but that's pretty draconian. Are you also no mercy, hard on crime, 1-strike and you're out person, or more progressive in that area?
You answered your own question. Seems you're having trouble taking responsibility for the fact that you acted unprofessionally by lashing out with presumptive attacks on my beliefs, which are incorrect. The culture of that workplace is a mitigating factor for the lower ranked staff, conversely it is a compounding factor for the senior staff that failed to put an end to it and discipline those involved.
Should you have been fired? Depends on how much you abused your privilege and what you did with that information. Your comparison to 1 strike laws and greater crime is pretty rich given the information you improperly accessed could be used to blackmail others, or whatever. It's important to treat such information with the utmost respect. Your cavalier attitude and inability to accept responsibility in this regard may be very common within the tech industry, but such attitudes are also why there is a growing movement to; take data out of the hands of companies, and to harshly punish companies who fail to protect the data on one hand while vacuuming up as much as possible with the other.
I'm honestly somewhat mystified at the lack of personal ethics where you could justify in your own mind that you shouldn't be fired for this sort of breach of trust. But I shouldn't be I suppose...I've seen all sorts of appalling behavior excused with 'gee, it's not like I killed anyone...'.
This kind of thing has to involve cultural change where people internalize on a deep level that looking celebs up is a big no-no. You have to judge people by contemporary standards, and for this reason I think you do not deserve as much punishment as future offenders.
It has mostly been reactionary because the cost of being proactive and that most of the active work would likely be associated with national security, which is often outside the public eyes.
My thoughts exactly, though it seems the incident happened a few years ago when the mentality of "move fast and break things" mentality was much more acceptable.
I'd be very concerned if there still aren't processes and technical barriers to prevent the majority employees from accessing user data. IP address, phone number, etc. should require top-level authorization to access - not something an engineer or even marketing/sales person can just look up.
>Seems like something one could build a SaaS business around- send them reports that <user> accessed <fields> about <customer ID> on <date>, along with a copy of attributes and roles about each user. Service could offer deep dives, querying, reporting, along with ML or rule-based flagging to say "That seems odd".
Wouldn't that just expose user data to an even wider group of people while doing this reporting?
Nothing sensitive about what you just described? Seems like with that info you can start making intelligent answers to security questions or possible rainbow table look ups.
I wonder if tokenizing all PII for storage, with a special lookup server accessible only to a small group, would reduce the threat of these kinds of attacks.
Obviously some things like SMS or sending of email need to manipulate this data, but rx/tx of eg sms given a placeholder token could also be similarly siloed, so most technical staff would never have or need access to raw PII like IP, phone, or email.
Unfortunately geolocation for ad targeting and other customization features is still probably going to make a “what country is this user in right now?” possible for most devs/SREs, which is itself a bit of a physical location change traffic leak, but making it near-impossible for most devs or SREs to view untokenized user PII would probably be a huge step in the right direction.
For quite a long time (and probably still today but I have no idea) there was at least as much monitoring and security infrastructure built around watching and auditing what Facebook employees did to data as there was around what third-parties might be trying to do to Facebook infra and corp networks.
It already exists it part of the process mining+ monitoring box.
This kind of system are already used to both optimize business processes and conformance check then by organizations like banks and hospitals.
Through tools are currently focused mainly on the use case for process optimization and regulation conformance checking less so for irregularity detection but tools like that exists to. I think to remember SAP has some form of self learning/calibrating irregularity detection "service"/tool.
So it's less about creating it then about spending Mony on it and from scratch up analysing your internal thread model.
It's quite expensive, some of this software is sold for higher 5 digit numbers even for "small" use-cases.
You're misidentifying the real problem which is not in the software itself. The real problem is that data needs to be accessed by marketing, analytics, sales, executive, HR, and other parts of the company all the time. It's a human problem managing the impedance of data access blocks... and guess what at most companies the block is seen as a significant cost far greater then the "cost" of PII violations, leaks, or hacks...
But go ahead and find a random group of 3 ex-googlers with no domain experience to raise ~$2MM and chase it anyways. Then when you burn through the money you can go back to your FANNG job with a nice raise.
Yes, full role-based access control and audibility is key. This also highlights the need for data masking built into these DB systems. If data is gold, you need vaults to protect it.
It seems to be something quite common at Twitter to give information from its users, I have seen interviews where people openly admit that you can find the identity of an account holder if you have friends working there. After the last "hack" it looks like once you are inside their system, there are not many safeguards or auditing.
It would be good if this were an exception but it isn't. The easiest way to gain access to lots of privileged information is to work as support worker for a bank or insurance company.
> A millenial himself, [MBS] spent his youth eating fast food, playing Age of Empires and first-person shooter games, and keeping up with friends on the internet, according to people who’ve known him since childhood.
It's worth remembering that dictators are not inhuman, and they are not so different from us.
> Asaker would pay more than $300,000 to Abouammo, deposited in a Lebanese bank account that Abouammo had a relative open for him. “Proactive and reactively we will delete evil, my brother,” Abouammo texted Asaker just before one deposit of $9,911.
They structured [0] the bribes to avoid SARs; structuring really does happen.
> A third, a Saudi, was “a professional” who used encryption to conceal his identity, though once he signed in without encryption, and Alzabarah was able to track his IP address.
> [Alzabarah] spoke with Asaker on an open phone line and communicated via email.
> So rather than follow the FBI’s request to keep things quiet to assist the case, Twitter lawyers brought Alzabarah in the following afternoon, accused him of improperly accessing user accounts, and told him he was temporarily suspended.
Operational security is hard. Just one slip-up can doom the entire scheme, and here we see those slip-ups from everybody; from the folks being targeted by MBS, from MBS's goons, and from Twitter.
> It's worth remembering that dictators are not inhuman, and they are not so different from us.
It's also worth remembering we're talking about someone who assassinates his critics and cuts them up into pieces. I would say the "bone saw" aspect outweighs the "playing AoE" aspect and he is very different from us.
100% this. It’s also important not to dehumanise history’s bad guys, because that leads to a culture of complacency whereby people think “it could never happen here, because they were monsters and we are not”.
The idea that morality is a function of situation more than character is called "situationism" in philosophy, I just looked it up, see [1][2]. Experimental psychology does find that character is fairly weak in most people and that under external pressure people will often behave inhumanely (Stanford prison experiment, Milgram obedience experiment, etc).
I still believe it is important to not be overly cynical. My mental model is that most people are neutral or good. It might be proved wrong say 5% of the time, when people are indeed put in MBS-like position. But the vast majority of the time it will be correct, and it will be much more useful than the cynical model where your normal friends are potential murderers that just lack opportunity. A more correct mental model is not always more useful.
Another very common example throughout history: cannibalism. Nobody thinks they're capable of it yet somehow in a starvation situation it just happens to happen.
My mental model is that humans are pieces of shit, each and every last one. There is no such thing as a good human.
By insisting that MBS is some sort of specially evil person, I feel that you are devaluing the importance of genuine ethical consideration. I do not disagree with your specific points about his misbehaviour, but I want to emphasize that it is the throne and crown which empower him to do so much harm, and not any sort of blackened and hateful heart.
This is super important. A lot of people think that bad guys always behave like monsters and you can easily recognize them. In reality you would probably be positively impressed by a lot of them if you met them and didn’t know their background. Hitler was able to convince a lot of smart people to follow him and I have read he was very personable. Bernie Madoff could convince . Trump/Obama/Clinton haters would probably be surprised how nice these people are when meeting in person.
Normal people don't personally send bone-saws out into the world, but look at the rhetoric surrounding the recent protests in the US and Hong Kong. Plenty of normal people (many mainlanders, many persons of a particular political party affiliation) are cheer-leading the rabble to get dispersed with the maximum possible application of force - and most of them don't even have any stake in either the issue being protested, or the impact of the protests themselves!
And assuming the police are normal people too, they are all too happy to carry this out.
Normal people are quite happy to endorse or inflict violence against the enemy. Most of us just aren't in a position of power to do anything about it, though, other than post on Facebook, and write angry letters to the editor.
But give that normal person power over other people and freedom from consequences... And, well, you get something quite repulsive.
Indeed. It's easy to forget that very different behavioral rulesets can create similar conduct in some situations but radically different conduct in others. For example, your dog doesn't operate on the same moral code you do. It may love you, enjoy seeing you, would never harm a hair on your head, and enjoys all the same activities as you. It might also catch a squirrel, slowly pull all its limbs off, and spend the afternoon frolicking with its corpse. Don't mistake someone sometimes acting like you for someone who thinks like you.
A lot of people can't understand just how often people, including their friends, lie; yes even to them! People think they will be able to detect any lie.
I get the impression that Twitter has become a much smarter organization through really painful growing pains like this. I still wonder how I would play it if I became knowledgable. How long does this go on, do you just keep moving the person along on their career ladder? It seems like you'd need help from the FBI in how to maintain the charade if it were to go on for too long. But fascinating, I'm sure it comes up.
Sigh. I wondered if someone would come in here and do that. The entire reason I did not paste it myself is because this paragraph has a much different effect on the reader after they have read the rest of the article and understand the backstory.
Edit: Thanks for all the upvotes. I'm glad some in the HN crowd appreciate quality over shortcuts and spoilers. (I was starting to get discouraged!)
I don’t like this article because it omits some crucial details that could lead readers down a specific path of thinking.
It’s unclear if the inside men, Alzabarah or Abouammo, are living on H-1Bs, full American citizens, or are in the process of immigrating. Depending on the answer to that question, Twitter may need to block the employment of immigrants or citizens to stop this sort of industrial sabotage in the future. Otherwise, every country will try to have their own inside man and Twitter will be forced to overdedicate resources to countering them.
If they’d made his citizenship status clear, the solution would be far clearer to readers.
Individuals are more likely to want to commit espionage when they have money troubles. Why not hire only the already-wealthy?
Individuals are more likely to want to commit espionage when they are tempted with sex. Why not demand evidence of membership in closed religious communities to address that?
On the former, that's absolutely a thing with all sorts of levels to it. Base level is that most companies run background checks including a credit pull. Next level is use of elite academic credentialing, i.e., "we only like to hire from ___".
On the latter, because lol that is not going to have the effect you're looking for.
> Depending on the answer to that question, Twitter may need to block the employment of immigrants or citizens to stop this sort of industrial sabotage in the future.
Seems to be a bad test. American citizens left to join ISIS, American citizens have formed cults, American citizens have mailed bombs to people. If a Saudi Arabian prince plopped a Hublot into some random kid's hands and said "give me an email," do you think their patriotism would prevent it? I don't think so, actually, didn't the recent crypto scam involve 2 Americans?
What you're suggesting sounds like some kind of throwing out the baby with the bathwater, but even more nonsensical.
I think a better conclusion is that companies should architect with the assumption that there's a mole, not engage in some kind of border-control nationalist purge.
That's only a bad test if American citizens should be trusted as hires? I think what he was saying is that twitter should avoid Americans and hire h1bs
That way the American government has done some trustworthy checks for you
Looks like Ahmad Abouammo (Twitter’s former head of Middle East partnerships) was arrested in Seattle in Nov 2019, but Ali Alzabarah's escape to Saudi Arabia was successful (at least in terms of being arrested by the US government):
https://www.justice.gov/opa/pr/two-former-twitter-employees-...
BUT, as of a month ago, a filing was made to drop the charges (?!):
https://www.theverge.com/2020/7/28/21345794/twitter-employee...
Fascinating case...