In the context of a start-up, cost is a big factor and then perhaps (hopefully) handling growth. You could start small and refactor apps/infrastructure as you grow but I am unsure how one could afford to do that efficiently while also managing a growing startup.
On the selling soul to cloud provider, I don't see it like that. I have a start-up to bootstrap and I want to see it grow before making altruistic decisions that would sustain the business model.
Once you are past the initial growth stage, there are many options for serverless, gateway, caches, proxies that can be orchestrated in K8 on commodity VMs in the datacenter. Though this is where you would need some decent financial backing.
(I am not associated with Amazon, Google or Azure. I do run my start-up on Azure.)
I'm down a similar route, but I must point out that beyond a certain number of users / scale, Serverless becomes cost-prohibitive. For instance, per back-of-the-napkin calculation, the Serverless load I run right now, though very cost-effective for the smaller userbase I've got, would quickly spiral out of control once I cross a threshold (which is at 40k users). At 5M users, I'd be paying an astonishing 100x the cost than if I hosted the services on a VPS. That said, Serverless does reduce DevOps to an extent but introduces different but fewer other complications.
As patio11 would like to remind us all, we've got a revenue problem, not a cost problem. [0]
Sadly, anything more in-depth than that, you'll need to sign an NDA with AWS to learn anything about the performance limits of their services (eg Redshift), and you won't get that unless you're already a big customer there. Azure's not going to be falling over themselves to let you know where they fall short, either. This is vendor lock-in, and is why there are so many free cloud credits to be had to startups.
This is also a reason I believe SaaS companies will find it is harder than they realized to arbitrage between clouds, and business models based on that may not be able to get that right.
High scalability always revolves around the storage layer. There are plenty of options in the free software world; MongoDB, Redis, variants of MySQL/Postgres replication, Cassandra, FoundationDB, and many more.
Using one of those is where you'll spend most of your operational time if you really need that level of scalability. Most people don't, but the options are there if you really need them.
If you are happy with your storage layer, which most people are, the rest scales horizontally pretty easily. And there are plenty of free things you can use to get what a cloud provider gives you.
The CDN is always going to be tough to replicate on your own. In the end, latency is bounded by the speed of light, so you can only bring your files closer to your users. I wouldn't expect you to build one of these yourself; just buy one until you're the size of Google.
Your app should be designed to scale horizontally; don't keep any state in your app, delegate it to your storage layer so you can scale a CPU-intensive app up across multiple servers.
There are quite a few API gateways around; Ambassador comes to mind but there are a million. I personally use raw Envoy for everything. I was load-testing my website the other day and pushed 5000qps through it from my cable connection before I decided "it's probably fine". (I started dropping frames on the Twitch stream I was watching, though ;)
There are plenty of "serverless" frameworks that emulate what Lambda does. knative comes to mind. I have not experimented with them in depth, but am intrigued by the idea. (I am more intrigued by turning config files into webassembly-compiled programs, to make existing apps more configurable at runtime. This is like serverless, but less general.)
> Add in Route53 for DNS, ACM to manage certs, Secrets Manager to store secrets, SES for Email and Cognito for users.
CoreDNS scales nicely and has an API. cert-manager is an open source way of obtaining certificates (though it's tightly coupled to Kubernetes); either ACME (letsencrypt) or your own root CA. There are a bunch of free software secret managers; Vault, bitnami-labs/sealed-secrets, etc. I personally use git-crypt ;)
Email deliverability is always going to be an issue. Like the CDNs, you might want to delegate it while you're small. Use anything except Mandrill.
