Tips for running HBase on low-memory EC2 (opens in new tab)

(dev.hubspot.com)

18 pointsgsabo11y ago2 comments

2 comments

Thanks for writing this. We're also running a large HBase cluster on EC2 and frustrated with the poor suitability of instance offering (there's literally nothing cost effective for HBase in the current generation).

I am curious why you chose c1.xlarge over m1.xlarge, which is cheaper and has double the memory. In my experience HBase tends to be more memory bound than CPU bound.

bbeaudreault11y ago

We did initially use m1.xlarge, a few years ago when we first started with HBase. I no longer have the hard numbers available, but while I agree HBase is memory hungry, our m1.xlarge servers were constantly pegged for CPU. For a while we ran both types, and eventually phased out the m1.xlarges entirely.

We run pretty frequent compactions due to our read and write patterns, so of course if you do not need to compact as often you might get much better mileage out of the m1.xlarge.

j / k navigate · click thread line to collapse

2 comments

necubi11y ago

I am curious why you chose c1.xlarge over m1.xlarge, which is cheaper and has double the memory. In my experience HBase tends to be more memory bound than CPU bound.

bbeaudreault11y ago

We run pretty frequent compactions due to our read and write patterns, so of course if you do not need to compact as often you might get much better mileage out of the m1.xlarge.

j / k navigate · click thread line to collapse