The unusual thing about this market is that by constantly improving the price point of their product but keeping their profit margins the same, AWS has forced every competitor in it to actually compete at the limits of their capability - so any scheme like this that got any serious traction would presumably be self-undermining. The economists will have considered all of the unused provided capacity in the model before they priced it. Sorry to be miserable ;)
I don't know why you think so. AWS is not very price competitive. I make good money consulting on setup of more cost-effective alternatives to things like S3.
The large cloud providers sell on brand recognition, convenience and trust, not cost. They may compete with each other on cost to try to cannibalise each others markets, but they're not even close to pushing the envelope on cost efficiency in terms of the prices they offer.
Just wondering, what is more cost effective than S3 with the same reliability?
If you have your own servers I can see running zfs and replicating snapshots every few minutes to a remote machine but if you aren't, what else is competitive?
I do agree that a lot of AWS services aren't cost effective though, especially at any sort of non trivial scale.
Setting up your own with any of Ceph, Gluster, Riak, Swift or similar depending on your specific needs. Heck, I've got setups where we've been served well for years with a combination of inotifywait, grep and rsync (combined to trigger rsync instantly on modification events). It really depends on your access patterns, and there are lots of potential savings from making use of domain specific knowledge for your specific system.
In general, you can beat AWS with 3x replication across multiple data centres even with renting managed servers, especially as your bandwidth use grows as AWS bandwidth prices are absolutely ridiculous (as in, anything from a factor of 5 to 20 above what you'll pay if you shop around and depending on your other requirements). Lease to own in a colo drops the price even further.
Anything from 1/3 to 1/2 of AWS costs is reasonable with relatively moderate bandwidth usage, with the cost differential generally increasing substantially the more you access the data.
Most people also don't have uniform storage needs. If you use your own setup, people tend to be able to cut substantially more in cost by reducing redundancy for data where it's not necessary etc.
See also the Backblaze article that mentioned Reed-Solomon - if your app is suitable for doing similar you can totally blow the S3 costs further out of the water that way.
Compared to what? Honest question - not trying to be cheeky.
If you are comparing against a self managed solution, are you factoring all costs into that equation (fully burdened labour costs, disposal, cooling, power, etc.).
For a reference point: The hardware cost for storage servers with full redundancy is in the order of $0.10 per GB (plus electricity, rack space etc.). Buying your own hardware and throwing it away every year is price competitive with Nearline (and you get full online storage, no three second delays and slow retrieval).
Of course there's good reasons not to roll your own solution until you reach a certain scale, but it's certainly possible to compete with Google and AWS on price.
> For a reference point: The hardware cost for storage servers with full redundancy is in the order of $0.10 per GB (plus electricity, rack space etc.). Buying your own hardware and throwing it away every year is price competitive with Nearline (and you get full online storage, no three second delays and slow retrieval).
Well, it would be price competitive if you considered only the hardware costs and not, e.g., the labor cost involved.
Consider that there are labor costs around operating services on AWS as well, and it's generally more expensive by the hour, and labor cost of managing your own servers is fairly low.
I have plenty of servers sitting in racks that haven't been touched in 5+ years; if you were to operate on a "buy and throw out in a year" practice, my typical labour costs would average a couple of hours per new server for setup. Or if you rent managed servers, you don't ever need to touch (or see) the hardware.
Conversely, you can get bandwidth at a tiny fraction of the cost of AWS bandwidth, so the moment you actually transfer data to/from your setup, AWS gets progressively more expensive.
The unusual thing about this market is that by constantly improving the price point of their product but keeping their profit margins the same, AWS has forced every competitor in it to actually compete at the limits of their capability - so any scheme like this that got any serious traction would presumably be self-undermining. The economists will have considered all of the unused provided capacity in the model before they priced it. Sorry to be miserable ;)