The required either PayPal or passport. I have no PayPal account, and their 3rd party verification system only allows passport from your country of residence (signup requires providing a contact address and they pre fill using this address; you can’t change the passport country). I am a British citizen living in Japan, and therefore hold a British passport; there was no way for me to provide a Japanese passport. I asked what I should do to comply, and they banned my account 6 hours later.
I can’t be the only one to experience this, can I?
I’ve got streaming replication of my core data going from one cloud company to other company as that way if one has some antifraud system go rogue on me I still have access.
As somebody who used to spend a lot of time thinking about drives breaking it’s an interesting shift.
I had similar struggles with some non-IT service providers in Germany. They couldn't fathom why I have non-German nationality, German address and driving license from third country. Passport, German address and driving license all have different address on them (all three being EU addresses). It is apparently huge red flag in EU in 21st century. Incredible
This is actually exactly what happened with us. After creating a new account with the intention of exploring the ARM64 services, our account was unexpectedly suspended. I contacted them to have information on the specific concerns regarding my customer information and the reason for deactivating my account. Unfortunately, we did not receive any response to our inquiries.
Is this something new? I (Norwegian) have been using Hetzner for 10+ years, and never had a problem, and never had to attest my identity. CurrentlyI have a four servers running there. The last one was set up approx. a year ago, IIRC.
That may be the difference. Some nationalities get KYC'd easier than others and they seem to take it very seriously
Another thing to consider: cloud providers are not very interested in individuals as customers. They usually want companies as customers, that also buy more than a 3$ vserver. A solution for this problem could be a sign-up fee (50 or 100$), to pay for an extended manual vetting of customers, that is then added to the account balance.
A key theme in the "cloud vs data center" story is that most public cloud providers (AWS, etc...) were really easy to sign up, requiring a CC and nothing else.
Meanwhile, hardware vendors wouldn't even talk to you as an individual / small business.
I ran into the same thing as well, maybe my real name sounds a bit funny to their system? It was very discouraging to move forward after being instantly banned. I reached out to them how I could verify and the only way was sending them an unencrypted mail with my Passport copy. Upon request they suggested I could simply send them a fax.
Note, this has been some years ago and I've never gave Hetzner a new try. As long as I can see from professional experience, you will have a lot of back and forth with Hetzner support, which becomes quite bad the moment your team is international because they'll always manage to sneak in some sort of German text. It really feels antiquated having to go through support for basic server hardware debugging. Eventually they'll often resort to replacing your instance.
We ban these people — they're violating our ToU by engaging in illegal activities. But they come back. With different names, different IPs, different browsers, different credit cards. They have complete identities to burn. (We spot the correlations anyway, along other dimensions I won't disclose here, and so can keep them out pretty effectively.)
And guess what? Very often, their requests are coming from Hetzner IP blocks.
I don't think the scammers have a direct business relationship with Hetzner, mind you. I think Hetzner tries just as hard as we do to stop these people from making use of their services. But I believe that these Hetzner boxes are either set up as exit nodes of one or more common VPN providers; or they're being registered for other purposes by other parties, and then resold on the secondary market on dark-web forums.
If I were a VPS provider, and I didn't want to support illegal activity, I'd probably just give up on providing service to individuals altogether, only taking corporate customers; and even then, requiring a DUNS number or something as an additional proof-of-work for that corporation, so that people can't just keep spinning up corporations in places where that's essentially free.
Hetzner hasn't gone that far; but it makes sense to me that if a user account is flagged as needing extended verification, and the ops person responsible for verifying the account takes a look at the user-lifecycle activity logs for the user, and sees that this user has: their IP coming from multiple places during registration vs login, their browser locale and timezone bouncing around between requests and set for settings uncommon to the country their IP is originating from, etc. — that the answer would be "ban" rather than "ask the user why the heck that's happening."
One time out of ten, the user is a real person doing something weird. The other nine times out of ten, the user is a scammer and is going to make up some story about being a real person doing something weird. Every scammer has their very own pool of man-hours, and if you're in the critical path for their scam, they can burn a number of those man-hours being really insistent that they're authentic. Until you let them in, and see that they immediately start up the same dumb phishing-scam bot script that all the other scammers purchased.
They are remarkably well known for having draconian anti-fraud
Sounds like a good thing to do?
My current hoster in Germany made a surprise call and asked me what the name of the hotel near the address I provided was. This after I submitted the order and before they accepted.
So what's the point of all that? Seriously, one server with 16 cores and the memory bandwidth and storage speed of SSD drives should be able to serve many thousands of simultaneous requests and millions of users per month. I can't help but feel that the cloud infrastructure and microservice movement of the 2010s was.. a scam.
I just need a place to run Docker, similar to what we had with Linode for running a shell 20 years ago. And don't tell me how to set it all up. Just give me a turnkey Terraform (or Ansible-inspired declarative configuration management tool) setup that has stuff like load balancing and some degree of more advanced features like denial of service protection out of the box. What we used to think of as managed hosting, but open source, and with sane defaults for running standard suites like Laravel, Rails, etc.
I have to assume that I'm just terribly out of it and something like this already exists. Otherwise I can't understand why someone doesn't just offer this and provide a way for us to pay them the millions of dollars we lose to spinning our wheels on cloud infrastructure with 1% utilization.
You're just missing the whole point of why you need n>1 instances. You're focusing on what might be the least compelling reason for the majority of applications, which is scalability.
Reliability and fault tolerance is a major selling point, and you can't simply expect to have adequate performance in more than one region if you're providing your services out of a single box in a single data center. The laws of physics ensure your quality of service will suck.
Operations also compell you to have multiple instances, ranging from blue/green deployments to one-box deployment strategies. And are you going to allow your business to be down for significant amounts of time whenever you need to update your OS? How would you explain all that downtime to your boss/shareholders? Are the tens of dollars you save on infrastructure a good tradeoff?
Also, security is also a concern. You're better off isolating your backend from frontend services, and software-defined networks aren't a silver bullet.
And in the end, are the tradeoffs of not designing your web services the right way really worth it?
Similar with security. A simple monolith is more secure than isolated front- and backend.
It's also about simplicity. Many services come pre-configured on.AWS, with clustering, failover, backups, etc already present.
When you are small, you can sysadmin your server fine, and you don't yet need the cloud.
When you are colossal, you want everything custom, and the cloud would cost you a huge amount, so you go for your own servers (and possibly datacenters).
But for the midrange, the expense of hiring several more SREs to handle dedicated servers is usually higher than the entire AWS bill, and the cloud looks very reasonable.
Turns out, developers are cheaper than DevOps engineers and AWS budgets. So if a linear spike in servers necessitates a superlinear cost in operational complexity, maybe working highly performant web app backends is actually cheaper?
Most of the time we just removed GHz limits, sometimes adding vCPUs in advance, they fared well.
NB: IaaS provider with a guarenteered resources
I think the general consensus is that the most-root cause for why cloud infrastructure and microservice architectures matter is more People, less Technology. Its hard to hire highly experienced people, especially at the hundreds-to-thousands of engineers scale that many companies operate at.
That, in isolation, is reasonable. The actually uncomfortable question that no one wants to address is: was that a scam. Put another way, how much of that was a zero-interest rate phenomena, and how much "state of art best way to engineer" things was influenced from that, and has root causes in US monetary policy.
Between AI and high interest rates: if you're not ready to spend the 2020s questioning every single thing you think you know about software engineering; you won't be long for the industry.
Computers are so powerful now this may be the wrong instinct. You can mind boggling amounts of processing for say... $30,000 in hardware, and even with redundancy it would be much simpler to manage.
Indeed the main selling point of microservices is People, more specifically how to set hard boundaries on responsibilities and accountability while ensuring independent decision-making.
Another important selling point of microservices is flexibility. It's trivial to put together an entirely new service from scratch, written in whatever technology crosses your mind. With microservices you don't even need access to a repo.
And yet it’s got basically perfect uptime. Whereas the impenetrable cloud goes down every week and costs billions a year.
Some monitors indicate that hackernews has a 99.84% uptime in the past 30 days. Hardly "perfect".
Are you kidding me? Prices, sure, but comparing the configuration burden using AWS or GCP alphabet soup to a dead-simple Heroku setup is ridiculous. You set up something like hirefire.io for autoscaling, you connect your Github repo and that's it. You're done. Compared to weeks worth of headaches setting up Terraform Cloud and CI actions and massaging your state file and spending days tearing your headout over insane obtuse NAT configuration parameters? It's literally night and day.
I'm glad to have learned AWS, although I don't know that I'll build another cloud server. I think the biggest problem I found is that services tend to require each other. So I ended up needing an IAM for everything, a security group for everything, a network setting for everything.. you get the idea. A la carte ends up being a fantasy. Because of that interdependency, I don't see how it would be possible to maintain a cloud server without Terraform. And if that's the case, then the services just becomes modules in a larger monolith. Which to me, looks like cloud servers are focusing on the wrong level of abstraction. Which is why preconfigured servers like Heroku exist, it sounds like.
An open source implementation of Heroku running on Terraform on AWS/GCP could be compelling. Also a server that emulates AWS/GCP services so that an existing Terraform setup could be ported straight to something like Hetzner or Linode with little modification. With a migration path to start dropping services, perhaps by running under a single identity with web server permissions (instead of superuser). And no security groups or network settings, just key-based authentication like how the web works. More like Tailscale, so remote services would appear local with little or no configuration.
Also this is a little off-topic, but I'd get rid of regions and have the hosting provider maintain edge servers internally using something like RAFT combined with an old-school cache like Varnish to emulate a CDN. The customer should be free from worrying about all static files, and only have to pay a little extra for ingress for the upload portion of HTTP requests for POST/PUT/PATCH and request headers and payloads.
Oh and the database should be self-scaling, as long as the customer uses deterministic columns, so no NOW() or RAND(), although it should still handle CURRENT_TIMESTAMP for created_at and updated_at columns so that Laravel/Rails work out of the box. So at least as good as rqlite, if we're dreaming!
Edit: I don't mean to be so hard on AWS here. I think the concept of sharing datacenter resources is absolutely brilliant. I just wish they offered a ~$30/mo server preconfigured with whatever's needed to run a basic WordPress site, for example.
For $400 you can buy an SSD with 1.5 million sustained IoPS.
It is no big deal to build a 1TiB RAM box these days.
The cloud almost always ends up costing a lot more at scale and increasingly at the beginning.
In some part this is because the cloud is someone else’s computers that you have to help pay for.
The current incarnation of cloud was designed in part to work around scaling issues with Linux networking, which have long since been solved. What a $5 Linode/VPS can do today vs 10 years ago is much different from a linux standpoint alone before additional horsepower.
At the same time tooling to host (docker, ansible, terraform) is making traditional devops heavier at times.
Unfathomably based.
Having some sort of vpc where you can safely put your db server that only listens to requests from servers within the same vpc (e.g., web server) sounds like good practice to me.
All the Linux distributions I got to know use sensible defaults so that critical services don't bind to a public-facing interface / bind only to localhost, e.g. mariadb and mysql on Debian.
Besides that, Hetzner's "Robot" interface allows to configure which ports/IP addresses you allow access to your Hetzner server.
Your data is not encrypted in rest.
Their security is not bad but someone can easily plugin whatever they like.
Try this at Google. Google not just has much stricter physical access control but also FULL control over the hardware. They know the firmware of the Mainboard. The server don't even unencrypt and start if you move them out of their Datacenter.
Google doesn't just has ingress egress they have undersea cable.
There is so much difference between hetzner and google it's not the same thing.
It's up to you to set up "encryption in rest": E.g setup LUKS on a Hetzner machine.
> Their security is not bad but someone can easily plugin whatever they like.
That's not how it is. Check e.g. this six-year old security video from Hetzner: Hetzner is great but it's not a good choice for companies which really care. Your data is not encrypted in rest.
Their security is not bad but someone can easily plugin whatever they like.
If you only compare the cloud products. How do you know that they aren't encrypting their drives?
And BTW, they own undersea cables as well ;) https://www.hetzner.com/presse-berichte/2016/01/156818
Please educate yourself before you make such wild claims.
Research costs are also very expensive for x86, with 2 vendors (Intel and AMD) competing savagely in many different markets such as high-end gaming, servers, ultrabooks and low cost PCs. However, ARM (the company) focuses on licensing the designs and instruction sets. So their research costs are split among many licensees while ARM focused on what it does best: designing chips.
That in turn translates into many vendors who can focus on improving manufacturing efficiency, procurement, logistics and sales. x86 vendors have a wild range of manufacturing lines to keep up.
Finally, the laws of supply and demand play a huge part. The ARM market is 2 orders of magnitude (!!) greater than x86 as far as raw number of processors shipped, ~370M vs ~32B chips in 2022.
> the number of transistors tend to be similar between the two platforms
sorry, come again?
Quick Googling shows:
- Intel W9-3495X has 56 cores for $5889
- AMD EPYC 64 has 64 cores for $4299
- Ampera Altra M128-30 has 128 cores for $5800
Hetzer offers a 16 core AMD (v)CPU machine at 225% of the price, so that roughly works out.
16 cores doesn't say much. It depends how many DMIPS those cores will yield.
Also, x86 backwards compatibility and SMT don't cost much in terms of power or transistors but both have large costs in of verification work.
AWS has been trying really hard with Graviton 3.
Building your own based on them isn’t really difficult at all; docker buildx works remarkably well and build tools such as maven, sbt, etc. seem to have rather decent support for building multiarch images.
Even better if you have decent build automation, implement cross building for amd64 and arm64 once and just make sure your FROM images support those.
ARM design had to be adjusted/developed for a cell phones often cost 10% of a normal computers (main use of x86). I can buy a cell phone a display, battery, memory and ARM cpu for $100 total (no idea how much ARM chip alone cost). The phone will have a semi-acceptable performance.
Even cheapest x86 CPU with emi acceptable performance probably cost more than $100 and cheapest x86 cpu more than $50.
Nvidia claims to be working on getting in the server game later this year so maybe ARM will get to x86 levels of competition.
But according to my Minecraft benchmarks in Oracle's instances, its better than old Skylake cloud instances... by how much is hard to say since other tenants generate so much variation, but there are proper reviews of the whole SoC floating around out there.
On my workload, using Intel as a baseline. AMD under performs relative to standard benchmarks. ARM over performs relative to standard benchmarks. I have seen good performance from Ampere, particularly for the price.
However, ARM architecture complicates my workflow significantly. I've found that they are not worth using it right now due to compatibility issues, easier to just stick with x86 for most things. Dealing with required missing packages just kills the value.
My system isn't generally compute limited anyway. IOPS and egress dominate rather than MIPS.
We've moved on from lakes!
Hetzner offering them right now is curious. Maybe its a pipecleaner for the next Ampere gen?
Minecraft is limited by a single thread, but some stuff does run on other threads. In addition the JDK will do compilation and GC on other threads now, which is especially apparent if you run GraalVM like you should.
Minecraft wont scale to 16 cores. But if you buy a 2C/4T instance, you are going to get less performance than you would get on a wider one.
The latest Xeon and EPYC offerings are very performance competitive, I doubt we need to overhaul and entire processor ISA paradigm for continued improvements.
So, if you want to compete, you'll have to go ARM. Amazon graviton and Apples M series have only made it more appealing.
The main reason to switch is cost.
Currently anyone can make an x86-64 processor.
Nice to see that things seem to start moving a bit now. Energy consumption and cooling are major concerns for data centers. ARM is just so much better when it comes to energy consumption. So the potential should be there.
That doesn't seem likely, as the release of AWS Graviton based instances predates the Apple M1 release by about two years
We have seen performance improvements in PHP, Java, and various CPU-bound tasks such as video transcoding.
This goes for either EC2 m7g vs m6i and also for Fargate x64 (from my experience, perfomance seems comparable to m5 instances) vs ARM (Graviton2).
Well if they're spending less to power my ARM machine, then they should damn well be charging me less to host it.
That's the best, and only, long term path towards more efficient compute.
https://hastebin.com/share/biyejiriyi.bash
https://hastebin.com/share/unocorikef.bash
https://gist.githubusercontent.com/12932/8ba27254846072a43b0...
https://gist.githubusercontent.com/12932/8b47af122ba79c79b82...
sysbench cpu --threads=2 run
Intel events per second: 1864.20
Arm events per second: 6687.05
Coremark
Intel iterations/sec: 20077.633516
Arm iterations/sec: 23625.767837
To offer a somewhat more varied example I've tested the POV-Ray benchmarker on Debian 11 on Oracle's Ampere Altra servers ("A1.Flex") versus an identical setup/build running on a 2 GHz EPYC 7281-based x86-64 VPS, and on that single-threaded test the Arm VPS handily outpaced the EPYC with almost 2x the performance.
dd if=/dev/urandom of=1GB.bin bs=64M count=16 iflag=fullblock
Intel:
Compressor Compress. Decompress. Compr. size Ratio Filename
memcpy 5062 MB/s 5013 MB/s 1073741824 100.00 1GB.bin
zstd 1.5.2 -2 2083 MB/s 4743 MB/s 1073766410 100.00 1GB.bin
zstd 1.5.2 -5 210 MB/s 4775 MB/s 1073766410 100.00 1GB.bin
zstd 1.5.2 -9 85.5 MB/s 4774 MB/s 1073766410 100.00 1GB.bin
Arm:
Compressor Compress. Decompress. Compr. size Ratio Filename
memcpy 10876 MB/s 10950 MB/s 1073741824 100.00 1GB.bin
zstd 1.5.2 -2 3175 MB/s 11168 MB/s 1073766410 100.00 1GB.bin
zstd 1.5.2 -5 192 MB/s 10967 MB/s 1073766410 100.00 1GB.bin
zstd 1.5.2 -9 146 MB/s 10909 MB/s 1073766410 100.00 1GB.bin
rsa 512 bits 0.000070s 0.000006s 14268.2 159885.6
rsa 1024 bits 0.000405s 0.000021s 2468.4 47757.7
rsa 2048 bits 0.002847s 0.000078s 351.3 12830.1
Hetzner x86: sign verify sign/s verify/s
rsa 512 bits 0.000067s 0.000004s 14893.9 240902.8
rsa 1024 bits 0.000127s 0.000009s 7845.9 114300.9
rsa 2048 bits 0.000874s 0.000027s 1144.6 36800.6
I have no interest in subjecting myself to this treatment.
What I think you’re suggesting is that my trouble is with the machine—some version of frustrated human fighting with inanimate objects. In general this is a framing I oppose; automated machines are created by people, and people carry the responsibility for their creations. “The computer did it” is never an excuse.
So in a way, yes; I tend to view “error messages” as you put it as an extension of their creators, who very well may be worthy of indignation.
Ran the kernel build benchmark (result is seconds, lower is better):
AMD64:
272.916
273.128
270.477
ARM64:
1011.799
1004.713
1015.261
Also ran sysbench cpu --time=60 --threads=4 run
AMD64: events per second: 14681.70
ARM64: events per second: 13455.11
sysbench memory --time=60 --threads=4 run
AMD64: 5859951.00 per second
ARM64: 6052749.14 per second
Next up I timed compiling nodejs. time make -j4. I ran the test two times and took the faster result for each.
AMD64:
real 28m46.385s
user 107m48.971s
sys 5m12.994s
ARM64:
real 39m18.443s
user 146m25.801s
sys 7m53.271s
Rescaled the ARM VM to 8 vCPUs:
real 22m20.624s
user 162m30.176s
sys 8m50.104s
What I learn from these benchmarks is that you might get some really good value out of these ARM instances if your usecase is not impacted all too much in terms of performance.
Personally, if I were to do it, and I had some sort of load-balanced application that uses multiple backends/frontends/whatever, is balance across the two and compare over time.
And to be 'fair' you'd want to compare similar pricing.
Or if you had a consistent group maybe try a Minecraft server and move it between the two every week and see what people "feel"?
That depends on how many vCPUs are they running on each physical core for both x86 and ARM.
This will never be 100% scientific or correct since there are many factors that come into play, stuff like noisy neighbour for example.
What I'm trying to get is a ballpark idea of how their arm vCPU compares to the amd equivalent.
With all the AI hype I'd assume they'll try to offer GPU instances again with precautions put in place.
Also haven't been sued yet. But let's see.
But I don't really want to do business with Oracle anyway, maybe it's better so.
I'm currently running Ubuntu but am open to alternatives. My local Raspberry Pi uses a fun, but little maintained OS called Hypriot which is just this: Start Docker and not much more.
CAX11 - 3.95eur/m
CAX21 - 7.19/m
CAX31 - 14.39/m
CAX41 - 28.79/m
It's been a wonderful ride. It's cheap and reliable.
I wish our team was big enough to afford having to deal with dedicated hardware.
Been using a couple VPSes in Oregon, pretty happy with them.
* Oregon
* Virginia
Unless I'm blind, the price clearly says 4.52 EUR/mo with IPv4 at https://www.hetzner.com/cloud for a CAX11 instance, the smallest listed there.
Edit: nevermind, I didn't deselect Germany's 19% VAT. It is indeed 3.79 euros before sales tax.
---
Intel 1-core VPS single: 719
(2 GB RAM)
---
Intel 2-core VPS single: 729
Intel 2-core VPS multi: 1402
(4 GB RAM)
---
AMD 2-core VPS single: 1141
AMD 2-core VPS multi: 2214
(2 GB RAM)
---
Ampere 2-core VPS single: 872
Ampere 2-core VPS multi: 1716
(4 GB RAM)
---
Ampere 16-core VPS single: 880
Ampere 16-core VPS multi: 12143
(32 GB RAM)
---
And for some fun, my iPad Pro from 2018, single: 1135
If Android CPUs are 4 years behind Apple, then Arm on servers is maybe 8 years.
elastic aws cloud billed by the second is for great good. glad to see this continuing to mature.
neither of these is great when they are idle 90% of the time. one of them is really not great.
why are there so much idle yet paid for resources? it’s complicated, and not for technical reasons.
empty housing is similar.
But their hardware design could improve. See picture below.