Again, as the parent poster also posted, I think you have never worked on large data. Twitter is like a big mailbox, only that every mail only has 160 bytes. This has been solved 10 years ago.
If you don't understand that the request distribution matters more than payload size, you aren't even seeing the problems.
I encourage you to analyze infrastructure for a twitter style app using inbox duplication. Once you model this against hardware costs you'll learn something about how utterly expensive write amplification is in a hot data set that must be backed by ram due to availability requirements.
Wait, see my comment below. Twitter received 15B (yes, B) API calls/day last July. How does that compare to your typical email client?
I don't want to argue that Twitter is astoundingly hard, but serving ~170K requests/sec can't really be that trivial, even if they're 160 bytes (they're not, since Twitter sends metadata, logs those messages, tracks service metrics, etc. for those messages)
If you treat twitter like a big mailbox, things will work "ok". It's not the worst approach ever, that's for certain. But end-user perceptible performance would be a fraction of what twitter has today.
P.S. How many images does twitter serve up per day at present? That's a tad more than 160 characters of data.
Another key difference is that email users generally contribute directly to their provider's infrastructure costs in providing email as a service. Email infrastructure (and the user experience) is fragmented, and global funding generally scales with global load.
Instead twitter must monetize via advertising of some form, and so the percentage of folks who do not respond to ads acts as a really strong factor in your cost calculations. In this sense, email software has it easy, and can be extremely wasteful in the resources it consumes.
It's not just that the availability expectations of twitter are higher than email, it's also that the economic base of the infrastructure is far more sparse.
And yet another key difference is that there are few email server installations that support half a billion users. Saying that the scaling problem is "solved" because all you have to do is copy, say, gmail, is kind of silly.
We use play in all our projects and hit some bugs as well. (Like different versions of classes being in jars and in code -> signature mismatch)
It's a great project and no software is without bugs.
He means that if linkedin would be sold for 60 million, they would get the first 60 million. Then afterwards the normal shareholders are served until the initial ratio is reached.
There is no point proposing them to use dedicated servers as already many did. They are 100% sure dedicated servers cost more, so let them keep their ec2's and let them fail!
I wonder why google, yahoo and facebook don't run their site on ec2... if it's cheaper.
I think failure is looking more and more likely the more I read of their blog and details of how they run things.
It's a very interesting case to analyze though - perhaps acquired too soon, never had enough pressure on monetization until it was too late... Questions over how well the site was architected. Sounds like they're using a lot of 'new' unproven 'hip' things. Casandra? :/
Seems like the founders and YC have been extremely quiet about the problems... It'd be interesting to hear their take on things.
Also can't imagine how Conde Naste could be happy with things.
Google et al. do essentially run their servers on EC2. However it's their own version of EC2, in their datacentres. They have a whole pile of virtualised servers they can turn on an off by the minutes. They have internal accounting systems so that each team is 'charged' based on what they use. Google are so big, so it's cheaper for them to make their own datacentre than use someone else's (i.e. Amazon's).
EC2 only came about because Amazon run their servers on it, and they had so much spare capacity, so they sell it.
>Google et al. do essentially run their servers on EC2.
If by EC2 you mean "bunches of servers", sure.
On GoGrid you can buy cloud servers, or you can rent dedicated servers (and you can intermix the two). The latter are quite a bit less expensive for a given quanta of resources, while the former obviously offer greater dynamic flexibility (with a significant premium).
Actually considering the terrible I/O rate of services like EC2, dedicated often offers a dramatic advantage.
>They have a whole pile of virtualised servers they can turn on an off by the minutes.
But they don't. So they use none of the upside, and have all of the downside. Yay!
> But they don't. So they use none of the upside, and have all of the downside. Yay!
That's not true at all. We shut down machines when we are over capacity (rarely) and we often have to bring up a bunch of new machines where there is a traffic spike.
> I wonder why google, yahoo and facebook don't run their site on ec2... if it's cheaper.
Well, for one, they are A LOT bigger than us. But you'd be surprised who DOES run on EC2. The biggest one I'm allowed to tell you about is Netflix. Their entire streaming service is run off EC2. I guess they're idiots too, huh?
Well that's hardly a fair point. It's going to depend on the scale of your traffic, the type of site, the networking/sysadmin manpower available, tolerance for failure, etc...
EC2 certainly isn't always cheaper, but it also certainly can be cheaper.
