undefined | Better HN

0 pointsm0zg7y ago0 comments

My networking is good enough for my needs: 10GbE copper. Not 100GbE, but 100GbE is hard to actually fully utilize without jumping through major hoops.

My business is mainly deep learning R&D. Current cloud GPU, networking, and storage pricing gives me ulcers given my compute needs and the size of my datasets.

I do run my website in cloud, with redundancy and all that. I also use cloud storage for backup, and for K8S registry. If I was selling e.g. inference services, I'd be running them in cloud too (passing the costs onto the clients). Most of my local workloads could easily be shifted right across to any decent K8S provider.

But the fact is, my lone rack has been humming along with zero unscheduled downtime for 3 years now. I can count several global outages in each of the three major cloud services in this timeframe (most of them during US work hours, BTW), so I'm inarguably better off with the setup I have now than I would be if I moved it all to the cloud. Not to mention it already paid for itself several times over even though I burn through several hundred dollars a month in electricity.

0 comments

3 comments · 1 top-level

kbenson7y ago· 2 in thread

If your needs are fairly static, then that's all the more reason to go on-prem if it's not too hard to set up initially. If you expect to need to upgrade systems on a regular basis because you need the extra CPU/disk/memory capacity, or switching infrastructure to support increased network utilization, that all gets very annoying very quickly.

Also, if you're sourcing components yourself or from a third party that isn't shipping specific configurations that have been tested by hundreds or even hundreds of thousands of times in other installations (i.e. not Dell), then be prepared for the possibility that you'll run into weird errors from the specific combination of mainboard, CPU, memory, NIC, RAID, etc used in a system that can be extremely painful and time consuming to to diagnose for RMA. I mean, if you're lucky, that will exhibit itself as an a crash that happens early and is easy to reproduce.

At this point, I see the premium you pay to large name providers like Dell/HP/etc as not just insurance that they will quickly accept and replace any possibly faulty hardware but also as the fee they charge for ironing out all the kinks of a platform so you don't have to worry about them for the most part.

It's totally possible to build your own hardware for cheap, but I no longer have the will to do so. To me, cloud services are like that, just to a lesser degree. I can handle the hardware locally (even if not custom built), but I'm finding less of a benefit to doing so as I get better at dynamically handling load (which has benefits in areas besides cost as well).

m0zgOP7y ago

I'm not arguing this is the best solution for everyone. I'm just saying it's better for me, from both the reliability and cost standpoint. Frankly I couldn't care less what everybody else is using.

I have just checked the pricing of what my approximate (GPUs are different in my case) setup would cost to run on GCP, and it totals up to $40K a month. Not a typo, I did not add any extra zeros there. That's not including storage or networking costs, and _including_ the sustained use discount. That's nuts.

kbenson7y ago

I understand, I just wanted to express how I think it's really specific to to the task at hand (and vent a little about annoyances of the past). CPU intensive and or GPU might be a bad case, but it also depends on how actively you utilize the systems. For some of my stuff, I have what's likely close to an ideal case, where systems are only needed for about 5 continuous hours each day, Monday through Friday. I can turn them up on start or work, and off 5 hours later, and pay a fraction the cost for a full time deployment in the cloud since other than disk I only pay for the compute used.

> it totals up to $40K a month

Yeah, it sounds like your use case is bad for cloud (not that I'm an expert), especially if you're able to get that all in one rack. It's a lot more useful if you have different layers that may need to grow at different speeds. For example, a front-end load balancing set of systems, a group of web servers to aggregate some back-end services for page display, different service clusters, and a set of databases (read-only replicas and master-master replication, etc). Both being able to scale some systems vertically (just allocate more RAM and/ro CPU to it) and other horizontally (just add more servers to a load balanced set) can be extremely useful, especially when the unexpected hits and you find you load is multiple times the normal amount all of a sudden and you need to provision now.

j / k navigate · click thread line to collapse