You don’t download the modules when your function starts. You include 500mb of dependencies in your 50kb lambda function.

The difference in boot time between 50kb and 500mb is quite significant.