The cold start times have been quietly getting better over time on AWS Lambda. I just checked some python Functions and I'm seeing cold starts take around 80ms on average for a simple function.
Hopefully they have a solution for connecting to RDS from Lambda without incurring the huge cold start time and other problems of having the Lambda inside the VPC. Ideally it would be something like PgBouncer with a security group that only allows connections from your Lambda instances.
GCP has a semi-hacky solution that provides the ability to connect to your managed dbs from your Cloud Functions; one would hope that AWS can at least match that soon.