Yeah, this kind of thing is part of what I meant when I criticised AWS encouraging VPC use instead of end-to-end security.
But off the top of my head, you could always use the firewall API from the lambda to open network access between it and the RDS when the lambda starts. (In addition to using certs or IAM security on your TLS connection to the RDS db)
I think with client certs work fine with SQL Server and Oracle too, with standard clients.
But I'm sure you can come up with imagined scenarios where you end up cornered to use VPCs. I get it, these situations may exist. I'm just saying they suck, not that you'll never have to resort to them.