On hetzner it all costs less than 100$ per month. Doesn’t this imply that it’s better to rent than to buy?
They offer some Dell Servers for those that really want them, but most of their servers have a custom mix of consumer hardware, server hardware and in-house hardware (for example they use their own racking system), optimized to minimize lifetime cost in a datacenter. For example most servers use datacenter SSDs, but consumer CPUs.
TANSTAAFL
1. Physical hardware is a useful abstraction.
2. At the end of a three year cycle, you can sell something you own. (This has associated costs of course).
3. Rents can go up. Terms and Conditions can change. Credit cards can expire or be cancelled. In other words, renting introduces a significant dependency.
4. You are your own most important customer. Statistically, you are not The Clouds's most important customer. When The Clouds has a fault, it gets resolved based on its business model.
Engineering decisions are specific to a specific problem and all of them come with tradeoffs.
"There ain't no such thing as a free lunch"
(I had to look it up, figure I'll save someone else yhe trouble)
This is huge and often overlooked or undervalued. Renting means you have nothing when the contract ends, buying means you have something for your money spent.
This is what you expect prices to be when a vendor is buying in bulk.
Cloud vendors should be like "Costco" (buy servers in bulk and can pass along those savings & this is what Hetzner does)
The trick AWS (and other cloud vendors have done), is charge "Uber Eats" pricing for Costco items.
For some Hetzner servers (especially from their "Server Auctions"), for domestic customers the cost of electricity alone, disregarding the hardware cost, would sometimes be higher than the rent Hetzner wants.
Also keep in mind that hetzner is mostly a beowulf cluster. See e.g. this press picture from their newsroom page: https://cdn.hetzner.com/assets/Uploads/IMG-0546-91.jpg
They do have dell servers, too. But don't expect to run on one of these nodes unless the machine type has Dell / PowerEdge in their name. This means the management capabilities are minimal (but usually enough). A fully licensed management card can do way more.
Their renting pricing is ridiculously cheap though. Hardware failure is more common than on cloud providers (see cluster hardware), support has been helpful in the past. Overall IMHO a good choice if you are price sensitive. But even then consider development vs. deployment vs. running (server) costs. Servers might make up less of the total cost of a service than you expect.
Therefore, I'd argue that if you require continuous, raw computing power, Hetzner is indeed cost-effective. However, the cost-effectiveness elsewhere really hinges on your usage patterns.
So if you look at a 2 year horizon and want to keep cutting edge (or prefer operating expenditures to capital expenditures) then it's cheaper whereas looking at a 5 year horizon the capital expenditure will pay off.
Hetzner is very bare-bones compared to regular clouds (a friend who was after performance but wasn't prepared for this ran into some issues when a disk died), so you need some procedures for backup/replication in place but if your procedures are in place then you can save big compared to the regular clouds.
Another video showing the custom racks and other infrastructure: https://www.youtube.com/watch?v=5eo8nz_niiM
However, the killer isn't the purchase price. You have to put it somewhere with a lot of bandwidth, UPS, generators and cooling. To use your Hetzner example, you can't get co-lo for less than $119. That's 14U, which is more than you need for one server of course, but then there are usually charges on top of that.
So yes, for small installations it's hard to beat renting. You need to be using several racks for the equation to tilt towards purchase.
Just incredible, response times are incredibly low and there is no cap on data. The few times I have contacted support they've been great. The only times I have experienced downtime is when I have stopped or rebooted the servers.
Can't recommend enough, especially if you are inside the EU and care about data privacy. The fact that they're a european company is of great value to me.
If you are new on server provisioning and want to learn how to setup a new server, I wrote a small guide for how to do it for node-projects (works for pretty much any web app if you replace the node.js stuff): https://deployjs.com/ (hosted on that server)
I thought that was weird (I mean no other cloud provider asked for it but okay). I uploaded anyway because I thought it’s in EU so they at least have better privacy laws so they may end up deleting it later from their systems but I immediately got rejected. Not sure why no reason specified. So now they have my passport copy page without me having an account.
Oh well, seems like either they have an overly aggressive spam filter that gets false positives or they don’t really like accepting signups from Asia
Maybe not the most customer-friendly experience, but I figure half an hour of signup weirdness is okay for a solid after-signup service in the long term, especially as that half an hour is likely to deter a decent chunk of fraud.
And don't you dare suggest in their forum that foreign tech is better than German, instaban.
Fuck Hetzner
But if you just need some cpu (to transcode some video, crack some passwords, what-have-you, run a game/videochat server) - consumer cpu models give you unmetered gigabit or 30tb/month 10gbps. So can definitely be worth it vs a VPS.
Sure, you can get a cheaper VPS, but given a VPS with the approx same perf it would cost much more.
Been using them since 2012 or something, and have never had a problem (that wasn't my own fault).
I've been using their services for some 20 years at various scales, and the only complaints I've had are that their shared mailservers occasionally get blacklisted.
That, and the UI of their KonsoleH could do with a bit of a refresh.
> No Datacenter Deployment. The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted.
I could never figure out why they allowed blockchain processing, given how wasteful much of it was and how it impacted card supply for years, but there you go.
TBH renting a big GPU (A100 or better) and getting the run over with quick is usually best. VRAM is everything.
It is, with ample RAM and the latest CPU.
But even so, it's still quite expensive. I have the impression Nvidia charges more from data center customers. So, for cards available to retail consumers, it's not a fair cost comparison.
With only 20GB, RTX 4000 has limited use... I was expecting they'd be offering something with at least 60GB at this point.
"After reviewing your updated customer information, we have decided to deactivate your account because of some concerns we have regarding this information. Therefore, we have cancelled all your existing products and orders with us."
Someone has a similar experience? I've tried it twice, filled everything truthfuly, valid credit cards etc. and no still the same.
I’d assume it’s something to do with collecting recurring CC payments from Indian cards. That’s inherently difficult.
I mean what explains this? It's a billion dollar corporation and I am having serious reservations about whether I would even enter my credit card info on this site.
It's awful.
I would still buy from them because their reputation trumps any concerns.
I think the last sentence you have there is pertient though, maybe don't go for Hetzner unless you can also afford to have a hot-spare (but for the performance/price you probably can compared to any public cloud). In my friends case he was boot-strapping but needed the CPU so he was really penny pinching.
(I've now tried running the Q4 mixtral which is 26GB. 18GB is on GPU, 8GB through CPU. Gets about 11 tok/s.)
Given how quickly things are changing, using more generic hardware (i.e. PCs and Nvidia cards) or leasing cloud based services might make more business sense in the long term, so investigating fallbacks with more control, like this server, are worthwhile experiments anyway.
Further, while I've been an Apple superfan for a couple of decades now, their recent attitude towards developers stinks, and I'm loathe to give them any more money than I need to right now, so I might be migrating away from the platform in any case.
https://www.hetzner.com/sb/#additional=GPU
Looks like currently there are only GTX 1080s on the server auction, I don't know if they normally have other GPUs available as well.
From the OP link, the GPU-Server GEX44 server line includes RTX 4000.
And the server only cost €184/month.
The one with the RTX 4000 seems to be a standardized line of products.
Are there many other providers for such environments?
20GB RAM, can fit 13B param models, and maybe some quantized larger models.
That card is about $1500: https://www.amazon.com/dp/B0C2KMXQYG
I have no idea where the NVIDIA RTX 4000 SFF belongs in the hierarchy of speed and suitability for "AI" computing.
How does this compare to a NVIDIA A800 40G? (if that is still considered good, I can't keep up)
To me the RTX part of the name sounds more on the consumer side but I am probably wrong about that as well.
I don't know why, never got a response...
I went with ovh.
Marketing still doing damage control, I see.
