More armchair quarterbacking. You don't just pick up and move to save a buck when you have a whole company running on AWS' specialized architecture. There's a lot of process and a lot of overhead that factor into that consideration. A lifelong Mac user, for example, couldn't seamlessly jump into Windows just because "it's cheaper".
Even if you ran on bare metal on colocation, I doubt you save more than 50%. That's time wasted that could be more fruitfully spent iterating on profitable ideas. Plus that's exactly why you took investment capital in the first place, so you didn't have to do all that fiddly stuff. It's always better to maximize revenue than to minimize costs. On that note, the cost of paying all those salaries is probably 10-50x the Amazon bill, but you can't really cheap out there either.