Direct uploading like this has been possible for at least two years. However, in our tests it turned out that Amazon was silently losing some files >100MB (and almost all files ~1GB).
On the AWS forums it was clear that AWS was aware of this consistent problem, but was not able to fix it, and not willing to document it.
They eventually released "multipart upload", which remains the only reliable way to get large files to S3. Unfortunately, multipart upload is nearly impossible to implement as a web app (short of resorting to e.g. Java).
>Unfortunately, multipart upload is nearly impossible to implement as a web app
I am right now writing a library to do just that. I already have a prototype working, and we'll probably open source our library once it's reasonably tested. But perhaps you know something I dont... is there anything in particular that makes it "nearly impossible"?
Sounds like my understanding must be out of date as of this year! (Which makes me very excited about your upcoming library.) My conclusion was based on research ~1 year ago finding no OSS or commercial software supporting it, and forum posts for those projects by developers sadly explaining why they just couldn't do multipart yet.
My bad experience with files disappearing was entirely with web-based POST requests, ~2 years ago. Large file transfer from EC2 to S3 was reliable, but our POST requests on slower connections (even those that were very reliable) would return with a false report of success.
Direct uploading like this has not been possible through XHR as CORS support was only added this past year. However, you could mitigate it by handling the CORS headers through a proxy, using flash, or a couple other methods.
Multipart is not at all impossible from a webapp (depending on your browser requirements). FileReader API is available across all browsers (except IE < version 10, which I admit is a big problem). For IE, you can resort to some other solution (perhaps Silverlight, or Flash?)
I will second this. At the bottom of this article there is a reference the article I wrote a while ago when Amazon released this feature. I was planing on doing multipart uploading at that time using the FileReader but there was a bug in the way S3 did CORS so I didn't want to continue until that was fixed. They fixed it and I never came back to it. Maybe be a good time to try it again. Resuming a partial upload seemed like a good win to me.
What? Why is it hard? I just wrote a web app a couple weeks ago to enable my co-workers to upload large files (.5-1.5GB) to S3. Like, I don't doubt you, I'm just curious why my experience was so different than what you describe.
At Simon's Rock, the whole entering class consists of students who just finished 10th grade. In my experience there, self-selected 16-yr olds given the opportunity and expectation to live like adults mostly rise to the occasion.
That advice comes a bit late, given you have just 6 months of high school left. For you I would say two things:
Most 16 year olds will be happy to do what is expected of them if they are treated as adults. The worst parts of being a teenage isn't that it is difficult but that you have no freedom and nobody takes you seriously.
Right, we'd have to get some concrete numbers for the drawbacks of HFT (if any) and then compare with the benefits. What's the societal value of having a $20 spread vs a $230 spread on a million dollar transaction? What is an acceptable trade-off spread value, how do we define it in the first place?
I was excited when this was announced, but I'm not sure what it's actually doing in practice. On my Android 2.3 phone with up-to-date Maps app, it always picks the first of three alternate routes, even if the others have shorter estimated times and no lengths of red.
I've compared Google Navigation on my Android to Maps on my wife's iPhone, and sometimes they pick different routes. After checking the traffic, our conclusion was that mine was choosing the route with less traffic, while hers was always choosing the most direct route. I think it just re-orders them so that the chosen route is always first.
It would be pretty neat to be able to see your overlap between circles as a big venn diagram or something of the like. I keep finding out that I have handfuls of mutual friends with people I didn't know shared any of my friends (does that make sense?).
All a Venn (well, Euler, really) diagram would do is show you friends common between circles. I agree that building the graph and finding mutual friends would be nice, though. I generally just go to Facebook and look at our mutual friends, and I'm sometimes surprised, but it would be nice to have a tool that did it automatically.
I wonder if the API has a "mutual friends" call, you could write that pretty easily if so.
Amazon handles my apostrophe without drama. One advantage to having an apostrophe in your name is that you can spot when people migrate their databases without paying attention, because the apostrophes tend to double each time. DirecTV is now sending my bill to a Mr. D''''''A…
Some systems will manage to accept the apostrophe properly, and then make life difficult by making up the collation as they go along.
At least 80% of the time someone tries to look me up in an alphabetical list, I have to try to convince them that no, I really am on the list, but that I might be found at the very beginning of the Ds, at the very end of the Ds, in the Ds where I'd be with no apostrophe (depending on how they handle alphabetization of punctuation), or at the top of the Fs (middle initial) or in the As (on lists where the D has been interpreted to be an extra middle initial).
Even better, I often wind up on these lists several times when someone decides that it's easier to enter all my information again rather than look for it in one or two more places.
Credit-card name verification seems to be a bit smarter; every credit card I have has a different variant of my name on it (and a lot of credit-card forms consider an apostrophe 'invalid' in a name even though there's one on my business Visa card), but I've never had a charge rejected because the name was not an exact match.
http://www.wunderground.com/climate/greenland.asp#IceFree