undefined | Better HN

0 pointsbcongdon8y ago0 comments

Pricing depends a lot on how much memory your job requires[1] and how much processing each record requires -- i.e. the pricing is more sensitive to usage.

As a very rough estimate, for a light-to-medium load of 1Tb, the cost would probably be in the ballpark of ~$0.50. AWS's own reference MR framework[2] (which is mostly a tech demo) quotes prices in a similar order of magnitude.

Corral isn't great for processing-heavy MR jobs, as Lambda pricing rises quickly if you need a lot of memory or take a lot of time with each record. But, for small-ish low-overhead jobs, it can pretty easily beat the pricing and hassle of using something like EMR.

[1]: https://aws.amazon.com/lambda/pricing/#Lambda_pricing_detail... [2]: https://github.com/awslabs/lambda-refarch-mapreduce/

0 comments

1 comments · 1 top-level

shaunray8y ago

Hi OP, I am the person that made the most recent changes to the AWS Labs refarch. I had been working on a golang version and wanted to clean up the python one. Sunil the original author used the AMPLABS benchmark to calculate the results table. I was planning on updating it with the 1 and 5 node test. Would be happy to include Corral as well.

j / k navigate · click thread line to collapse