Show HN: I wrote a book about using Lambda with Go

103 pointskevinmcconnell4y ago17 comments

Hi HN!

During the last few years, I worked on a few applications built with Go, running on AWS Lambda.

As I got to know the platform better, I started to find Go & Lambda to be a really productive combination. The applications were fast, and they ended up being much cheaper to run than what my team & I had built before. It’s probably not the best platform for _every_ application, but I was surprised at how much of our workload worked well on it.

As we brought new engineers on to our team and helped them get up to speed with the stack, I found that we were covering a lot of the same topics over and over — especially things like performance, testing and monitoring. Since this knowledge turned out to be very useful for our team, I decided to gather it into a book that will hopefully also be useful to others.

This is my first time putting something like this together, so I'm grateful for any feedback!

The book is at: https://kevinmcconnell.gumroad.com/l/lambda-go-book/hn (with HN discount ;))

There's also a link on that page to a free sample that you can download without registering.

Thanks!

Cheers, Kevin

17 comments

zxcvbn40384y ago

Golang is great for lambda - you compile in the entire lambda runtime and end up with a single 20MB executable that does everything needed. Other runtimes have a couple hundred megabytes of overhead for the lambda runtime and AWS SDK (though you don’t see the overhead unless your doing the ECR based lambdas). The part I’ve always struggled with is cloudformation only lets you inline 4K of code. The underlying api allows for payloads of up to 50mb before you have to pull from S3 or ECR, would love to see a little more flexibility - otherwise you have a lot of overhead to setup ci/cd pipelines and signal lambda to update from another source.

kevinmcconnellOP4y ago

What I've started doing (and how I cover it in the book) is using the CDK to build and deploy the Lambda code. The `GoFunction`[0] construct handles building and deploying the code, so you don't have to set anything up manually.

Underneath it still happens by way of S3 & CloudFormation, but the CDK abstracts away a lot of the details, which makes it quite convenient to use.

[0] https://docs.aws.amazon.com/cdk/api/latest/docs/aws-lambda-g...

kevinmcconnellOP4y ago

I forgot URLs don't become links in the submission text, so here's a clickable one :)

https://kevinmcconnell.gumroad.com/l/lambda-go-book/hn

tgv4y ago

Any experience with cold start times? I wrote a relatively simple lambda handler in Go, expecting it to be fast, but node seems to beat it. The Go version's memory usage is better, though.

kevinmcconnellOP4y ago

I found usually they'd be on the order of a few hundred milliseconds. They tend to be quicker if you have more memory allocated to your function (because the additional CPU power helps).

I've mostly found that my cold starts were slow enough to look bad in the metrics, but fast enough (and rare enough) that the impact on user experience wasn't actually that noticeable. Given the other benefits I was getting from Lambda (like the easy scaling and low maintenance), it was worth the occasional small blip in latency.

And for functions that aren't directly user facing -- like processing items from a queue -- I've not found it to be an issue at all.

Of course every use case is different though, and some apps can tolerate this more than others.

probotect0r4y ago

Somewhat related, Lambda has the concept of provisioned capacity. So you can have a certain number of lambda containers running at all times, which will help a lot with the cold start. Obviously this is more expensive so you will have to weigh the benefits for your use case.

rememberlenny4y ago

I have used Lambda extensively over the past three months in Python, Ruby and Node. The largest game changer for me was discovering Lambda Layers and realizing how I could avoid compiling zip folders to deploy. I’ve loved the iteration speed and performance, and recently have depended heavily on the Lambda + EFS combo.

I purchased the book and was surprised there wasn’t more on the EFS usage, but otherwise the book looks wonderful.

daenz4y ago

What you described with Lambda Layers sounds like a thin layer over Lambda container images[0]. What are the benefits to using traditional lambda + lambda layers vs lambda container images?

0. https://docs.aws.amazon.com/lambda/latest/dg/images-create.h...

rememberlenny4y ago

For clarification, Lambda Layers is the packaging of a node_module, vendor/gem, or python package folder, and then "sharing" that across various Lambdas, instead of deploying the shared files again and again.

The actual lambda file ends up being a few hundred bytes, because they are literally just one function in a single file.

There is no compiling involved in the Layers after its made once. The only changing part of the Lambda is the lambda_handler.

mbot53244y ago

My understanding is that a custom lambda image is the nuclear option; having everything fit cleanly into a normal lambda archive is the preferred option; and lambda layers are a type of middle ground if you've outgrown one but don't quite need the other.

bazhova4y ago

Have you found a scalable solution to manage layers with infrastructure as code? As soon as I introduced them to my team, they started demanding exploitations of the process and it became a mess. Would love to know the best practices! Also beware of EFS, our applications did some heavy lifting with EFS and we found a bug that broke our stack. Not sure it's completely production ready. Took Amazon well over a year to find it and reimburse :)

kevinmcconnellOP4y ago

Thanks! And that's a good point about EFS, it's something I've found very useful too. I'm planning to add some updates to the book, so I'll put it on my list of things to include.

Thanks for the feedback!

clmcleod4y ago

Just curious, what did you typeset the book with?

kevinmcconnellOP4y ago

I used Asciidoctor. It's mostly the default theme, but I tweaked a few things about it.

If you're interested in the details, I wrote a blog post that goes into it a bit more: https://www.kevinwmcconnell.com/writing/how-i-wrote-my-book

That post also has a link to a repo with all the settings/theme/etc, so anyone can take it and use it as a starting point for their own book.

clmcleod4y ago

In the article, you reference `asciidoctor-epub`—I think you mean `asciidoctor-epub3`? Might be worth updating or correcting me here.

1 more reply

clmcleod4y ago

Thanks!

j / k navigate · click thread line to collapse

Show HN: I wrote a book about using Lambda with Go

103 pointskevinmcconnell4y ago17 comments

Hi HN!

During the last few years, I worked on a few applications built with Go, running on AWS Lambda.

This is my first time putting something like this together, so I'm grateful for any feedback!

The book is at: https://kevinmcconnell.gumroad.com/l/lambda-go-book/hn (with HN discount ;))

There's also a link on that page to a free sample that you can download without registering.

Thanks!

Cheers, Kevin

17 comments

zxcvbn40384y ago

kevinmcconnellOP4y ago

Underneath it still happens by way of S3 & CloudFormation, but the CDK abstracts away a lot of the details, which makes it quite convenient to use.

[0] https://docs.aws.amazon.com/cdk/api/latest/docs/aws-lambda-g...

kevinmcconnellOP4y ago

I forgot URLs don't become links in the submission text, so here's a clickable one :)

https://kevinmcconnell.gumroad.com/l/lambda-go-book/hn

tgv4y ago

Any experience with cold start times? I wrote a relatively simple lambda handler in Go, expecting it to be fast, but node seems to beat it. The Go version's memory usage is better, though.

kevinmcconnellOP4y ago

I found usually they'd be on the order of a few hundred milliseconds. They tend to be quicker if you have more memory allocated to your function (because the additional CPU power helps).

And for functions that aren't directly user facing -- like processing items from a queue -- I've not found it to be an issue at all.

Of course every use case is different though, and some apps can tolerate this more than others.

probotect0r4y ago

rememberlenny4y ago

I purchased the book and was surprised there wasn’t more on the EFS usage, but otherwise the book looks wonderful.

daenz4y ago

What you described with Lambda Layers sounds like a thin layer over Lambda container images[0]. What are the benefits to using traditional lambda + lambda layers vs lambda container images?

0. https://docs.aws.amazon.com/lambda/latest/dg/images-create.h...

rememberlenny4y ago

The actual lambda file ends up being a few hundred bytes, because they are literally just one function in a single file.

There is no compiling involved in the Layers after its made once. The only changing part of the Lambda is the lambda_handler.

mbot53244y ago

bazhova4y ago

kevinmcconnellOP4y ago

Thanks! And that's a good point about EFS, it's something I've found very useful too. I'm planning to add some updates to the book, so I'll put it on my list of things to include.

Thanks for the feedback!

clmcleod4y ago

Just curious, what did you typeset the book with?

kevinmcconnellOP4y ago

I used Asciidoctor. It's mostly the default theme, but I tweaked a few things about it.

If you're interested in the details, I wrote a blog post that goes into it a bit more: https://www.kevinwmcconnell.com/writing/how-i-wrote-my-book

That post also has a link to a repo with all the settings/theme/etc, so anyone can take it and use it as a starting point for their own book.

clmcleod4y ago

In the article, you reference `asciidoctor-epub`—I think you mean `asciidoctor-epub3`? Might be worth updating or correcting me here.

1 more reply

clmcleod4y ago

Thanks!

j / k navigate · click thread line to collapse