Long story short, I've had an account with them for roughly 5+ years as I recall, good credit card on file, monthly bill around $20-50, US address. I use a few droplets to run dokku/docker containers for dev and 1 private Google Outline VPN endpoint, only for me. I only ssh into these from non-root and root should be turned off completely (but I'm only a hacker and not a F/T systems person).
Not sure if it matters but I don't do anything nefarious on these droplets-- no webscraping, no botnet creation, no crypto mining. I don't know what else would be considered verboten!?!
Try to log in with github, won't work and asks for email login. Try to login from there, and it asks for credit card info for "authorization" although the card is the one on file and being charged for so many years.
No recourse, and what a crappy way to treat clients even if they are small potatoes in the grand scheme of things. Thankfully I am building a new MVP all on AWS and have my company prod there too running for 5 years. So I don't have to put up with unprofessional dealings and major risks of service interruption.
Assume through no fault of your own that you could be shut out of everything. You have been warned.
[0] https://aws.amazon.com/lightsail/pricing/
A while back I made a stupid mistake while tired, and I was panicking and couldn't find the feature I needed in the account.
I called Linode at around 11PM CDT, and got a helpful friendly guy on the line who walked me through what I needed. I was amazed that I could get a human on the line in less than a minute after hours, when only paying $10/month.
This is probably the most reasonable risk mitigation you can fairly easily do.
I don't think there exists a vendor that's going to be foolproof. Most users aren't going to any have issues with any of the major providers, including DO, Linode, Vultr, Oracle, AWS, Azure, etc...
The problem is that customer "service" is handled by algorithms with no chance to reach competent humans who have the power to make reasonable decisions.
For work, I run a cloud service like this, and we do indeed have this kind of fraud. Every stolen credit card number under the sun has signed up for an account. People create thousands of emails (and brand new domains) to get free trials. And, when our systems identify them and we shut them down, they whine in all possible support forums. I don't think we've ever made a mistake, but that is likely due to the small scale of our operation.
The only thing that keeps me sane is the extremely poor opsec of the fraudsters. They have pools of thousands of compromised IPs they're using for the command-and-control infrastructure that they install, but they often mistakenly use the same destination crypto wallet (they have a little LD_PRELOAD script they install to try to prevent us from seeing this, but it doesn't work if you don't load dynamic libraries to read the process table), or use a predictable naming scheme, or taint accounts by accessing a few of them from the same IP. (I have a graph that connects the accounts through things like IP address, so this is often the fatal screwup for thousands of accounts all at once.)
I imagine that Digital Ocean has this same problem, but at a million times the volume. So, there are going to be mistakes. It's intrinsically difficult to establish trust at these small scales. DO probably feels pretty uncomfortable giving someone a bunch of compute resources when they haven't even tried billing the credit card. You would feel pretty uncomfortable paying up front. So this is where we land, sometimes you look like fraud and get killed with fire. (As you get bigger, you'll have weekly meetings with your cloud provider, and they probably won't shut you down automatically. But at that small scale, it's tough for everyone!)
DigitalOcean Killed Our Company: https://news.ycombinator.com/item?id=20064169
I would happily make a 20 or 50 EUR/USD payment in advance to avoid the risk. But AWS is not flexible enough to allow me to do so. I understand in some countries it's possible, but in EU there are no options.
I’ve found support response time from both so far to be excellent (UpCloud in particular though their feature set is just slightly behind Vultr and DO currently).
I had been testing the water with DO with a view to hosting some production stuff with them, but I’ve seen a few too many similar stories like this one and decided to get out of DO and rule them out for production stuff for now. Many of their direct competitors seem to provide much better support at lower levels of spend too.
What does everyone do to easily and automatically back up their side project droplets offsite?
It's supposed to be a last resort kind of backup as I also use DO backup solution (managed database) but seeing how they can close an account out of nowhere, it ain't worth nothing.
Every time I've messaged support for an account lock, 24 hours later I'm informed it was a false positive and the account is unlocked. Needless to say, this does not inspire confidence that my resources are going to be kept up and running past Kafkaesque fraud monitoring systems
Sounds like good customer service to me ;) I tried to open an account with Oracle cloud and it rejected my credit card. They never even answered to my messages.
Well, I was not overly convinced I want to do business with Oracle anyway, so now AWS charges the same card every month. Oracle seemed somewhat cheaper for the use case, but nothing that kills me.
It was blocked but DO unlocked it once I explained it in an email to them.
I am immensely relieved. They've done right by me.
Did they explain why it took over 48 hours to reply to you?
But somehow a droplet was part of a DDOS and network was booted off ( 1 network spike for 10 minutes). Let's just say it was stressful and very unwelcome at the time.
I also went to mention that i did receive feedback and there was definitely a human from their end involved.
But, I wouldn't want to imagine what would have happened if it were websites of clients. I took more measures by now, but i think the actual culprit was a contractor that did some work for me.
I'm wondering how other providers handle that ( a VPS for example)? Anyone has some insights on how it can be handled better?
Depending on what you mean by "booted off", any and every reputable provider will do the same. They all have AUPs that allow them to stop routing for your, at a minimum, if you're found to be sourcing traffic that's against their AUP.
That's all normal. You can't expect a company to keep you online when you're causing problems for them and the rest of the internet, and their contracts/AUPs are all worded so they can take action if your systems are found to be a problem.
What can you do to prevent this? Don't let your systems be part of a botnet or get hacked in general. More feasibly, stay up to date on all security issues and take multiple system backups so if you have to restore from a good prior backup, if that's possible.
> I took more measures by now, but i think the actual culprit was a contractor that did some work for me.
Could be, but my guess is not. Botnets survive on having massive numbers of servers in them, and are often automated exploits. More likely, you were running some exploitable software, and it was exploited. Updates need to be run, daemons often need to be restarted after updates because of older code still running in memory, etc. And if you were running some CMS like Wordpress or Drupal, well if you aren't doing it yourself and keeping on top of updates and new versions, you either better have someone on retainer to do it for you or give up and do something else, because you're a perfect target for a botnet at that point because those generally have problems on a regular basis (as well as the problems introduced by different modules they each might support).
The context is missing without my original sentence. "Network was booted off." Obviously not my account but "network access" and i could only connect through the console through their web dashboard.
> Could be, but my guess is not.
According to the files i found and the zipfiles he left behind. There would be a huge confidence of similar timing and folders involved. He even named the zip similar to the work he did.
I also know how botnets work, backups and security updates. I already took measures as mentioned before.
I never said and/or blamed DO for it. I'm not running sites on it, but some personal stuff as i mentioned ( again).
I just stated the situation as it was in my original comment and wanted some insights on how ( and mostly if ) it could be handled better...
Tbh. I handled your guesses in my original comment already at the time and you seem to be focusing on attacking my explanation of the situation instead of my only question and answering questions i didn't ask and already resolved. Which is really frustrating to respond too... As you're not talking to me, but a reader of the comments.
I am immensely relieved. They've done right by me. Kudos for a delayed but excellent response.
I'm looking for the next Digital Ocean. Something still bespoke, with higher cost but better customer support. So far oldschool VPS hosts are looking good.
I see value in supporting smaller companies (relative to large ones like Amazon, Microsoft or Google), but smaller companies usually tend to have better customer support when things go wrong, and are generally a lot more helpful. With DO, it has the kind of offerings by a smaller company with the kind of support by a large company.
I've been happy with DigitalOcean for a while now, but reports like this concern me, and I've also discovered after investigating that the reason MS-hosted email accounts (@hotmail.com, @live.com, etc) reject all emails from my VPS is because the entire provider is on the UCEPROTECT blocklist, and apparently has no interest in changing this.
This is insane, if I wouldn't have those backups and the same thing happened to me, I would be screwed. How come they don't contact you beforehand, or at least let you move out your data (keep the account open but cut the droplets from public internet if for whatever reason their algorithm detected nefarious activity on it, or a bogus DMCA, exploits happens, etc).
I wanted to try an alternative to AWS but look like I'll be back to it soon. Does AWS have a similar track record of closing account out of the blue? Is there any other alternative that have better support and would at the very least try to contact the owner before taking drastic action like this?
Luckily for us we added redundancy on payments and alerts, so I hope we won't be in the same case again.
