The government typically makes grandiose statements that sound impressive and make the accused sound like the devil himself but reality is almost always an order of magnitude more mild.
Unless I hear otherwise I'm gonna assume it was just pinging back to some C&C infrastructure he owned or something like that.
> Court documents[0] reveal Durachinsky wasn't particularly interested in financial crime but was primarily focused on watching victims, having collected millions of images on his computer, including many of underage children.
Well the relevant thing here is that he still had the pictures on his computer. Maybe he was taking pictures of 15 year olds when he was 14 or whatever, but they arrested a 28 year old man whose hard drive was full of images of people that are at most 17 years old.
This is a computer guy. Nobody would be surprised if he had spare hardware lying around from as far back as the 90s. If he just quick-formatted a bunch of spare drives and tossed them on a shelf years ago any forensics software would find what was on them before he formatted them. Considering that he's been collecting pictures since he was 14 I think it would be hard not to get him for possession if he's a hardware pack-rat.
That's a good point. I would probably forget about old drives as well - I thought they found the pictures on an active drive but realise now that this was a baseless assumption.
To be clear, I mean that the the way US laws are enforced, to the best of my knowledge, being underage yourself doesn't mean you are off the hook for having underage material.
I'd be interested to learn if this isn't the case.
That was a period when the only people talking outside the company were the ones not trying to fix things inside of the company.
Twitter never had dedicated machines for individual users. Its just not how the infra ever worked (at least until I left). Requests landed on random boxes behind load balancers, those boxes talked to pools of memcache or mysql, etc. At no point was there ever "racks" or "machines" dedicated to specific individuals like the article claims.
That being said, some users created crazy hot shards when specific tweets went absolutely madhouse, especially Beiber and those like him.
Random twitter internals tidbit: We had a unit of measurement called a "MJ". Its the number of tweets per second that we had when the rumors of Michael Jacksons death were circling. It basically overloaded the system and had us running around on fire. It was 465 tweets a second. Within a year we had crossed a line where we never were below that number again. Hence "we are at about 12 MJ's" was jokingly used to compare "hair on fire" to every day a couple of years later. =)
Sorry, we just kept the Bieber box hidden from you in the "special closet". It was a SPARCstation 20 and we were afraid if ops found it they would shut it down.
You know.. I found a special box (PowerMac Pro) in the "special closet" that had been moved like 5 times. It was Blane's old desktop and it had the twitter codebase checked out from like ~2007 era? Kind of nifty seeing just how much had changed since that got tucked away.
Before bring acquired, WhatsApp had what, 30 employees?
How did they do it? I know they used custom BSD servers so that a single box could keep close to 1M TCP connections open. I'm sure with a fixed target to aim for and all scope known upfront a small crack team of devs could do something similar for Twitter.
One-to-one vs. many-to-many messaging. The amount of work you need to do to deliver a WhatsApp message is constant and small -- just route the message to a single recipient's mailbox. The amount of work Twitter has to do to deliver a message grows as a function of followers. One celebrity tweeting another celebrity means you have to deliver the message to the mailboxes of the followers of both -- millions of times more work than WhatsApp per message. In addition, Twitter persists all the messages while WhatsApp doesn't.
This! Everyone keep's saying it's BSD Erlang and jumping on the Erlang train. Fine tools btw, but WhatsApp is super simple compared to Twitter. rolls eyes
I mean, obviously there must be a lot more going on inside twitter besides being a massive many-to-many pubsub messaging infrastructure. And frankly twitter can be improved by leaving a lot of stuff out and up to the client app, so it's not entirely unreasonabl?
> In one incident, he wrote, an error caused every user to log in as somebody else each time they refreshed the page.
Gotta admit, this story raised my eyebrow, this wasn't the dark ages of the web. What sort of crazy experimental authentication voodoo were they running?
Approximate counters are much much older than HyperLogLog. Here’s something from 1978: https://www.inf.ed.ac.uk/teaching/courses/exc/reading/morris.... Of course this solves a different problem because HyperLogLog is an, uh, interesting way to count followers (why would you need to do a count-distinct query? You can’t follow someone multiple times). In any case, the Flajolet Martin sketch dates back to 1985 and solves the same problem as HyperLogLog: https://en.wikipedia.org/wiki/Flajolet%E2%80%93Martin_algori...
Did they actually need to know Bieber's friend count both accurately and in real time?
I would expect that for top Twitter accounts like his, you could get a very good approximation by logging events that increment or decrement the friend count, and using the rates those events are occurring and the last known exact friend count to forecast the current friend count. That would not be exact, but I bet you could get it close enough that users would not see anything off about it.
Ha, I always joke that Twitter is a good example of why one should avoid distributed systems. Every often, you see it showing the wrong number of friends, tweets, likes, etc. I don't care, I suppose some people care about such metrics.
That's the essence of the CAP theorem. You can have two of three - consistency, availability, and partition tolerance. Sometimes it makes sense to choose availability over consistency.
A couple years ago, Bieber tweeted out a Twilio phone number for people to text. Luckily it happened on a Friday night, or it would have caused widespread outages.
Celebrity is an edge case that you have to be prepared for.
