I have a question (I've tinkered with Serverless stuff but never on a production app) but my underlying assumption might be wrong.
I like that the cold boot up performance for a single function is super fast when compared to a "Legacy Server". But after the legacy server is up it "stays warm" regardless of user activity. Unlike a lambda function which goes cold after a few minutes of inactivity.
So the first user that hits an endpoint after some inactivity has to wait (a few seconds) for the lambda function to cold boot vs. being served immediately on the legacy server.
Is my assumption here true? Or are cold boots on lambda super fast now? When I was doing this stuff ~8 months ago it would take like 5+ seconds to be served after a cold boot.
Servers will also eventually go cold, unless you pay for them 24x7.
With the Now 2.0 design, you instead are very likely to not suffer from cold instantiation to begin with, due to the healthy constraints imposed by the system.
In other words, instead of ignoring cold boots, you just want to embrace them and make them minimal to begin with.
You're right, they're slow. All the major providers are working on ways to speed up boot, and some users just send requests every few minutes to keep the functions warm.
The cold start times have been quietly getting better over time on AWS Lambda. I just checked some python Functions and I'm seeing cold starts take around 80ms on average for a simple function.
Hopefully they have a solution for connecting to RDS from Lambda without incurring the huge cold start time and other problems of having the Lambda inside the VPC. Ideally it would be something like PgBouncer with a security group that only allows connections from your Lambda instances.
GCP has a semi-hacky solution that provides the ability to connect to your managed dbs from your Cloud Functions; one would hope that AWS can at least match that soon.
I like that the cold boot up performance for a single function is super fast when compared to a "Legacy Server". But after the legacy server is up it "stays warm" regardless of user activity. Unlike a lambda function which goes cold after a few minutes of inactivity.
So the first user that hits an endpoint after some inactivity has to wait (a few seconds) for the lambda function to cold boot vs. being served immediately on the legacy server.
Is my assumption here true? Or are cold boots on lambda super fast now? When I was doing this stuff ~8 months ago it would take like 5+ seconds to be served after a cold boot.