Presumably the author would do much better with a VM or something from OVH, they'll just shut you off or limit you before it becomes a problem (not that they would care about 30 TiB).
For someone coming from my perspective, that would be a huge and unpleasant surprise to be billed $2600 for a service that I know costs $180 elsewhere.
Edit: especially so because people keep repeating the mantra that one should use the cloud to save money by not paying for what you don't use. Obviously, my 3 Xeon servers with 64 GB RAM each are way overpowered for sending out some GB of static files, but I wanted to have a bit of redundancy. But with my setup, there should be plenty of obvious inefficiencies for "the cloud" to eliminate
=> It feels like cloud should be cheaper than my dedicated servers. But it's not, and that is the unpleasant surprise.
Seems to me S3 is more designed for highly reliable data storage. If you wanna store a bunch of data in a way that's highly reliable and resilient to hardware failures and also not have to worry about managing RAIDs and clusters of servers for the data size, S3 is just the thing, and probably priced pretty fairly.
It's also a capable and flexible service. It's possible to use it for things it isn't really designed for and have it behave pretty well. Well enough that you can mostly ignore that the other thing isn't really what it's designed around. The "mostly" is important, though. If you start hitting any extremes while using it in a different way, then you certainly can run into some pathological cases in the pricing structure and pay way too much for something that would have been cheaper in a service dedicated to that.
The most common services like S3, Kafka, RDS (pg and mysql), Redis should be enough to cover most use cases.
With k8s and a dedicated smart team, it would be a possible adventure.
Moreover it can work great also on-premises. Several old medium business have their own physical infrastructure and they are not yet ready to move to the cloud.
But it's totally unfair to say they're "overcharging". The pricing is set up to encourage using the service properly. For example, did you know you get unlimited free (very fast) transfer between s3 <-> ec2?
You are free to use S3 however you want, but the pricing is set up such that people use a proper cdn backed by s3, do as much as they can between ec2 <-> s3, etc. instead of making s3 the backbone of their public site.
If you want to use it in a way it's not intended it will cost you more $$ which is how it should be.
But I guess simply running Debian on some server and keeping that up to date just isn't cool anymore. Everyone running a blog about their dog needs Cloudflare, S3, heroku, micro services and docker. Obviously with no limit on what kind of bill they will generate should something go wrong. You can just vent on twitter and generate enough attention that the vendor will make an exception for you and pay you back to limit the negative publicity. Who wouldn't prefer that to the tedious work that is maintaining a Linux install on a server?
Now that said it looks like his job is in cloud evangelism too so I’m sure a large part is that he wants to maintain a personal account to learn/hone skills. I’d recommend anyone do that, getting a look at aws without whatever your company is doing on top of it is pretty awesome for learning. Just don’t host massive S3 images and shoot yourself in the foot ;-)
The problem is that, at least in the case of public clouds, there’s a real risk of your bill exploding. I guess the author learned a lesson here, but I don’t think it’s the right attitude to start blaming the guy for “it makes no sense to tinker with AWS services” over here. Learning is probably one of his goals.
As for Cloudflare, hosting multi-gigabyte files is not what their service is for, I can’t see how you can blame them for having a limit on how large files they cache.
You survived the DDoS, that's great, now go see your bill.
The article doesn't use the term "piracy", but I'm curious what Microsoft's license says about public redistribution.
That's actually what I love about these dedicated providers. Often I prefer to be surprised by cutting the service instead of a gigantic invoice. For some business-critical applications I can understand the need to have it scale (and price) accordingly, for many other use cases it is better just to have the serve switched out when the traffic limit is hit off.
I really hate long-form articles that feel the need to explain the weather, someone's clothing, or their family member's eating habits.
COME TO THE DAMN POINT. This is the Internet. 99% of content is garbage, and I'm not interested reading through pointless content-free filler that could be generated by a neural network just to find out if your article somewhere contains useful/interesting information.
My hypothesis is that the client connected to Cloudflare and performed a HTTP range request for a portion of the 13.7 GB file. For an unknown reason, Cloudflare did not preserve this range request to S3 as the origin. It transferred the entire file, returned the range requested by the client, and dropped all bytes transferred into /dev/null because it is not caching.
The end result is that Cloudflare pulled down 30 TB of data while delivering 67 GB to clients in the one month period shown in the screenshots from the blog post.
See also https://blog.vbgn.be/2019/06/20/nextcloud-cloudflare.html
What's not stated here, and the OP learned, is that requests for larger files are always passed through directly.
https://support.cloudflare.com/hc/en-us/articles/200172516-U....
If you literally put a massive file on the public internet, what the actual fuck are you complaining about? The quality here is such shit.
Let's snipe at amzn, everyone hates them because they're winning, because of hn readers dependence on AWS to run their shit web apps... It's like: ope! It's been a day since we had a "amzn scammed me" post. Nevermind that the people complaining are either a) too ignorant and footgunned themselves, b) minimizing or lying about their own complicity, or c) outright scammers themselves.
You rent a car, you don't know what the total is going to be. You go to the hospital, you don't know how much you're going to have to pay. You book a hotel and don't know the total until you check out. You go to a restaurant and even if you order just one thing and saw the exact price on the menu, that's not going to be the total. You go to the grocery store, see all the prices on the items, add them up, and then when you go pay, surprise!
In some ways, yes.
In other ways, it's insane that people are able to borrow 5-10x their average annual income for an item (the house) that they have very little expertise in analyzing. In that sense, it's a process that is surprising it works at all.
You spelled "profitable" wrong.
Your bank should be upfront about their lender fees, points and origination costs.
Third party fees are either fixed or a simple percentage of the sale price.
The rest can be tricky, but should not be a deal-breaker for a new homeowner. Basically you are just paying the expenses for a short while up front. Interest through the end of the month, real estate taxes through the end of the quarter, Home owners and mortgage insurance and real estate taxes for the escrow account to cover 2-3 months.
[1] https://www.ftc.gov/news-events/blogs/competition-matters/20...
The FTC article you cite is actually in favor of the type of transparency lamented by the comment you are responding to.
> The staff comment explained the risk that the latter type of transparency might harm competition by enabling competing providers to coordinate or collude on price
Where "latter type" referred to "plan structures and contracted fee schedules between health plans, hospitals, and physician service entities." (The "former type" was "actual or predicted out-of-pocket expenses, co-pays, and quality and performance comparisons of plans or providers" which is what would effect parent's experiences; the FTC "encouraged" that type of legislature.)
I don't necessarily agree with the FTC here, but their comment isn't covering the lack of information that causes consumers have no idea what the bill is until they've already incurred the bill.
I’ve been using fixed price services (they exist) just so I don’t run into what Chris did. They aren’t evangelized as well.
In my opinion, it's a very deceiving practice that could definitely be fixed if businesses really wanted to fix it. But there's no incentive, because if you show your prices including tax and fees, they'll seem more expensive than the competition.
In the case of hotels, my experience is that with a lot of them you book online, pay the total, but then when you go check out, they've added some extra fees. Sometimes this happens before booking, you browse the options and choose based on price, but when you go check out, the total has changed because they've added not only taxes but some other fees as well. My most recent experience having this issue was with AirBnb just two days ago.
I think I'm missing something when people use the excuse of "taxes are different in different places" to say "we don't show taxes out of habit, or because it requires us to change our signs".
I deal with reputable rental companies. The rental price is based on demand but everything else is a known quantity. I’ve never had the quote priced higher than the online quote. And in a few cases I’ve had it go down by using certain credit cards or loyalty programs.
Unless you have scripts in place to nuke everything (which you will have to develop yourself since aws doesn not supply them), you have to manually login, and try to shut things of, while bill goes chaching
The only suspense left is whether I get the compact I signed up for, or some monstrous SUV because they’re all out of more sensible options. A few hints about how bad I am at backing up anything larger than a Camry has gotten results.
The last time I booked a rental car or hotel, I paid exactly what I was quoted at booking time.
[1]: https://www.ovh.com/world/dedicated-servers/rise/rise-1/
If my blog or apps are slashdotted (or HNed or whatever we should call it nowadays) they just load slower, degrading gracefully, and never stop or return an error.
If they didn't refund him, maybe he'd jump to a different provider, maybe people would see this and not sign up for AWS in the first place, etc. By refunding him, they likely keep a customer, don't scare away other potential customers, etc.
I loathe the day when I'll have a similar problem but, as I am not a Twitter user, I won't have the luxury of getting my problems fixed.
So they just waived it as it's better press.
Through I'm surprised they waved all of it, I would have expected half or so to be waved but then it doesn't really matter for them.
It seems to be pretty common not just for Amazon to be nice in case of such mesup's so that it is less likely to scare away people.
TL;DR Individual feels wronged, pushes things off AWS at their employer, AWS loses much more than $3k refund.
The author had an incident starting June 23rd, but didn't know about it until he got his bill July 7th, that's potentially ~14 days someone could have been abusing his account. A billing alarm would have reduced this to hours or minutes.
I would be interested how you suggest stopping a bill before it happens otherwise.. should AWS disable your website because you got posted on hacker news and now have a bill over your $10 limit? If AWS needs to stop your billing at $10, then it might need to shut down your EC2 instance and destroy your data...
If course they could offer the ability to configure the response, but if someone doesn't take the 2 minutes necessary to put a $25 billing alarm in place, what are the chances they will go through the effort of service/object based abuse policies?
At the end of the day, the issue here was that the user posted something online that people could abuse.. I don't think any CDNs cover 30 TB in free tier...
Maybe that should be an option?
Probably not a good idea for a business but if you're just using AWS to learn and otherwise fool around with, there's a good argument that it would be nice to be able to have a hard circuit breaker for at least stateless services.
I've actually heard people argue that they consider everything they put up on AWS is ephemeral so a hard circuit breaker should have the option to burn everything down but that seems like it would create its own set of problems for many people. Disabling data egress and EC2 seems as if it would go a long way towards stopping most of the unexpected bill stories.
And, yes, I'm aware of billing alarms and even setting up Lambda functions to take actions but, especially if you use S3 to host files, it would be nice to cap expenses at mostly your storage costs. I was doing some research for a very small non-profit that needs some hosted storage and I think Backblaze B2 is a better choice for them for this reason.
AWS does not do enough to prevent harm, even if a user is diligent with alarms.
Do i get now a higher liablility because the alert alerted me and i now need to be aware of it?
so i'm going on holiday and i'm fucked?
You know, a billig alert is a nice thing, its not the solution. Host a few things on AWS, tell me your endpoints and your bill just might be already in deep shit before you read your bill.
YES!!!!, if you ask them to do so.
$10 is a arbitrary small amount, but what if it's a web app and we are now speaking about $1000? And tbh. for many especially young people just $100 alone is much more then they can afford.
Or better have also a option for a dynamic hard limit based on a multiplier compared to normal traffic.
Even besides private persons there is a good reason to have a hard limit for a lot of businesses (or parts of them). Especially if it's on non-essential things people don't rely on which have a limited budged. (Like a lot of things provided for marketing only.)
Heck even for payed service providers a hard limit can make sense given that it's high enough, as a form of last defense against intentional or accidental DOS like situations (I mean in times of auto-scaling some DOS situations became cost explosion situations instead).
The problem with billing alerts is that (assuming they work and get delivered in time and are set to an sensible value):
- You need to read them in time.
- You need to be able to react to them in time.
Both NOT trivial in for private persons and also small companies which just got started (or do mismanagement or are to tight on budged).
Like what happens if you get it when you are on a party? In the cinema? Asleep? In the hospital? On holiday currently outside of reception?
Somehow the idea that you will react in just a view minutes and then fix the problem, too (or shut the service down). Is a bit to optimistic.
Through yes in the given case he might have ended up with "just" a view $ in the best case or ~600 or so in the worst case. Still I think he would have been happy with service discontinuing once the price reached a view hounded $ (let's just arbitrary say 250).
Sure, you can set up billing alarms and all, but the point is, for personal hobby projects, it shouldn't be this easy to screw up.
I've used AWS extensively professionally at my current and previous 2 or 3 companies, and I've had a personal AWS account but recently decided to close it just to be safe. I actually wondered if I could just remove my billing credit card, so I contacted AWS support about it, and they said you have to have at least one primary card on there. So the only option was to close the account. I only use a mix of Digital Ocean and Netlify and Heroku for personal projects so I might as well shut down my personal AWS account just to be safe, and I did. It's just not worth keeping one open, the risk is too great.
That said, unless that spike happens in a single day, you should be able to at least set a budget alarm to warn you. I think you should also be able to trigger shutdowns from those alerts within aws, but I never did.
I can see how it would be helpful for startups though or for services that are providing a free tier to users.
> Cloudflare was the least helpful service I could have imagined given the circumstances. A long term user and on and off customer thinks they were attacked for two days and you don’t lift a finger?
> File this under, “Things I should’ve known but didn’t.” Did you know that “The maximum file size Cloudflare’s CDN caches is 512MB for Free, Pro, and Business customers and 5GB for Enterprise customers.” That’s right, Cloudflare saw requests for a 13.7 GB file and sent them straight to origin every time BY DESIGN.
I don't really see how Cloudflare has much blame here. He's an "on and off customer" which I'm guessing means currently "off". They only cache a limited number of file extensions (qcow2 isn't one of them), and it's all documented.
AWS always seems pretty generous in resolving these cases at least.
In the long term this is a brilliant plan because it helps prevent people from blacklisting the provider.
Imagine someone gets hit with $3k bill on their personal, feels wronged, goes to work and makes effort at their employer to move off AWS.
I don't know about most HN readers but I'd probably fall in this category and past places I've done work for were +$100k/month corporate bills with AWS.
It's happening in this article too: Cloudflare wasn't in the wrong here, even the bill wasn't from Cloudflare, and the author is already publicly advocating against them. I feel quite uneasy about that.
Is it possible to pay them more to increase the limit?
Logistically I know this is hard for water or power, but it should be feasible for cloud computing. But I think this is an area where it’s not in AWS’ interest to set up that kind of billing control.
Which services you want to turn off or alter use of as you cross various thresholds vary, so what you probably want is a billing information services that sends alerts at configurable levels that trigger programmed actions that are more complex than a simple shutdown.
It would be easy to stop all spend at a threshold, but that means your entire set of apps stops working and all retained data (which usually has an associated periodic cost) vanishes irretrievably, which might be okay for an account being used for only toy apps but is going to be a business-ending error for any serious account
There's a good reason why cloud providers don't provide a simple “nuke your account at a particular spend threshold” option, and the fact that people who haven't thought things through think that they want such an option is actually a factor in not providing it.
If people can't keep track of spend with alerts to know that they need to take some kind of controlled restriction of services, they are going to regret unexpectedly having their whole set of services and data nuked more than an unexpectedly large bill in most cases.
I would lime quotas for individual services like “only spend $100 for EC2 and shut everything off when I hit it.”
My point is that I would rather have my stuff nuked than get a $1000 or $10k or $100k bill.
Setting up notifications and triggers is sort of possible but 1) that requires a lot of work for something that should be built in, I think; and 2) notifications aren’t in real-time so by the time I get a notification that I’ve gone over my $100 threshold I might already be at $1000.
I can compensate for this with scripts and third party services, but this would be so much easier if built in.
Unix has had disk quotas for decades right? Imagine if sysadmins left it up to users to monitor and control their usage and just charged overages. It’s so much more work for the user than when the system does it, or at least offers it.
My Linux host offers bandwidth quotas with similar cutoffs. I would never want an “unlimited “ quota where I got billed by the transfer and it was up to me to turn off. Some may want that, but not me.
That is simply not true for every use case. We were using it for various backup jobs, until a misconfigured one racked up a big bill. This particular job was a second backup. I have no problem at all with that failing! Now I am scared to let the various engineers and consultants that make up my team use it, because I can't afford the time to check personally what they are doing.
What you want is billing alerts, so you can be notified if usage is suddenly (say) 2x usual.
While billing alerts can help you against this kind of (semi) accidental overage, it won't do anything if the account gets pwned. The first thing the pwner is going to do is turn off the billing alerts.
A hard quota that required manual reentering of the credit card number (preferably in conjunction with 2fa) would help prevent that.
First what do you scope it to: Data, CPU, API calls, one or multiple projects. This needs to be designed carefully.
Then what do you base it one: Money or the unit used (e.g. data volumen). The later is much easier.
Then how to you technical implement the trigger. With many technical solution scaled to Amazone like services it's hard to stop at an exact limit, either you do some interpolation and maybe shutdown a bit before the limit is reached or you do trigger the shutdown when the limit is reached but then slightly overshoot it.
Lastly how do you do the "shutdown" on limit.
But in the end all is quite doable. Just not trivial. But not super hard either.
The article says they would not have helped but it doesn't say why... Maybe because it's delayed by a day?
But let's say you are:
- on holidays
- sleeping
- sick
- on a party
- etc.
Having setup alerts would still have reduced the bill.
Still assuming that something like 2h pass before you can react to an email is quite reasonable which would still have been ~150$ on the big day, which is ~6x of the normal _monthly_ cost in 2h...
And that is assuming the alerts are send real-time, which they are not.
And guess what, I didn't write a blog post about it. I just went to support, said remove the charges, they identified the services that created the issue so I could kill them, and they removed the charge.
Look at that, no fan fare. I had no emotion about it whatsoever. Maturity.
I'm not in my ideal financial circumstances, I just wouldn't have freaked out over that. I just would have handled it. And then blown them up in public forums if things didn't go my way. Customer support in the 21st century.
ew.
I'm not in my ideal financial circumstances, I just wouldn't have freaked out over a surprise $2000 AWS bill or a $190,000 AWS bill, fully intending not to pay it if I felt my activity did not warrant that.
There are no absolutes in guidances of behavior, only consequences. And the consequence here is potentially having your AWS account deleted in a month or two, with several remedies in between.
> Besides, some of us want to read tech drama :)
So maybe I should write a blog post about it.
https://www.cloudflare.com/terms/
2.8 Limitation on Serving Non-HTML Content
[...] Use of the Service for serving video (unless purchased separately as a Paid Service) or a disproportionate percentage of pictures, audio files, or other non-HTML content, is prohibited.
So 500MB limit or not, the author is already violating CloudFlare's terms of service.
Terns of service are also legal "CYA" documents. If CloudFlare was actually serious about that restriction, there'd be a technical limitation in place that would, for example, serve a 503.
if you look at the marketing is appears to be with out limit, now "unlimited" anything when it comes to technology should always through a red flag but CloudFlare as used this position to corner the market for CDN and DDOS services.
I think it is deceptive to hide these limits of the service deep in the ToS which they know every few people actually read
I'm the sysadmin for a small nonprofit so my organization qualify for the $3500 yearly azure credits but the inability to set spending limits makes me not use azure. If I make a mistake or worse an admin account is compromised the azure bill is potentially infinite.
With azure I think you can mitigate the issue somewhat by having a superAdmin account without limits and set quotas for everything else but I still don't feel at ease with that.
My organization is a music festival so the infrastructure really has to work and not stop one day per year. I can keep an eye on everything during the festival and monitor spending. If things stop not during the festival people are a bit annoyed until I can look into the issue but nothing bad happens.
In a perfect world I'd like to have a way to setup a limit where you need to go through support to increase it. I'd really pay for that. I think it's really too easy to receive a nasty surprise bill.
What’s odd is the touch points are cold. Ticket system support, phone call back etc. it feels like it’s going to be robotic canned replies but they figured out a way to make the people on the other side smart enough to understand the issue, empowered enough to do something about it, empathetic enough to want to resolve things “fairly”.
The problem is that I was utterly unable to talk to a human at UPS. I even went to a UPS Store but they were powerless to do anything.
The thing is that Amazon's automated chat bots and so forth just kept referring me back to UPS.
4 years later I've yet to buy another tube.
...
> I don’t feel like archive.org should be my site’s dumping ground since it can turn a profit if it gets popular. archive.org is a stop-gap for two files for the time being.
I'm trying to understand... he has decided to burden a charity with his distribution expenses?
In this case they'd be getting stuck with a pretty big bandwidth hit.
He could donate $10 to the internet archive and they ought to be even. Way cheaper than those $2656.74
He published a 14GB file and one day there were 2700 downloads resulting in ~30 Terrabyte of traffic.
He had the file behind CloudFlare, but since CloudFlare does not cache files larger then 512MB, all the traffic went to his S3 bucket and Amazon billed him $2700 for that.
Bandwidth on Oracle Cloud is $0.0085/GB with the first 10TB free each month, so this would have cost only $170. Alternatively bandwidth on Backblaze B2 costs $0.01/GB, but is free out to Cloudflare, so this traffic would have been completely free.
However it means sometimes things like this happen where a product’s incentives (serve any content at any cost) are wildly misaligned with a huge percentage of users needs (I’d rather my site, or preferably just the costly resource, be down than pay $2k).
There’s endless tuning non-enterprises can do to get our ideal behavior: but that’s the difference between pre-cloud and post-cloud computing. It used to take monumental effort to build high scale high availability systems. Your $5/mo Dreamhost site would just die under load instead of charging you thousands. Now enterprise use cases are supported by default and it takes careful tuning to opt out.
The only thing that could happen is they cap your data transfer at some point. But there are cheap VPS providers out there offering several TB of gigabit speed traffic and throttling instead of a hard cap when you reach your limit.
Point being their product was targeting me and designed appropriately. I forget the details but I know there were caps that were ample for my meager needs but would prevent this sort of accidental overage.
My point is that compute has become a commodity like electricity but without the built in fuses. My residential box can’t pull industrial amps.
IF your Services doesn't has a proper limit, you do make yourself suddenly liable to a much higher risk than before and you have to be aware of this.
It is the same shit when you rent a car: Do NEVER rent a Car without proper insurance.
I'm working with GCP professionally and i have used AWS in the prev company. I do ask my manager if i can use it to try a few things out and its fine but i will not put my credit card behind an account with unlimited cost risk (its limited probably but you know what i mean).
And its not even simple; Everything costs you money. Storing data, receiving data, pushing data, making api requests etc.
And what i find always quite surprising: How often people, even on hn, present simple file based apis where you can upload images and edit them or upload files and download them again or offering free services and that with AWS as a backend.
I just might be to long in this industry to see all those pitfalls of exploits and risks everywhere but i have the feeling that obvious respect against cloud service billing is neglected by most.
It's really easy to justify paying $1 for a small upgrade while you're playing. And only afterwards you notice that those $1 added up and have financially ruined you.
In the same way, $0.08 per GB (the effective price in the article) sounds really small and easy to justify. And we forget how they can accumulate...
You want to destroy a small startup with a free alpha version and AWS (or similar) backing?
Sure go ahead and send them tons of _legit_ (looking) traffic. This will first mess up their bill for this month and then mess up their statistics for the next month (when all the user they got disappear at once)...
And the alternative is paying someone to lock you into their ecosystem.
Are we really that lazy?
- Flexible scaling.
- Using other existing services from the cloud provider.
- Not needing to manage any hardware. (Or to hire someone to manage the hardware for you).
- Running your things in a data center. (Well connected, reasonable fault tolerant.)
- Less upfront operational cost (you are not required to pay for expensive hardware upfront).
For larger companies, especially such with massive amounts of micro-services it all about the flexible scaling and not needing to manage hardware.
- that is an antifeature
- seems trivial compared to keeping up with all the cloud gotchas
- possible anyway
- sure, but you probably don't need expensive hardware to start, or go with VMs
https://www.reddit.com/r/aws/comments/g1ve18/i_am_charged_60...
But this makes it even more scary in a certain way.
I mean what if they don't waive it in your case? You have no guarantees. Only the potential to cause bad press can save you.
That is not a very healthy situation I believe.
https://www.digitalocean.com/docs/spaces/#bandwidth
Digital Ocean may not be the best cloud platform but it's fairly cost effective.
Not only AWS is very expensive but also rather hard to use and all their forms and services pretty difficult to navigate as well, it put me off the cloud hype for very long time until I actually discovered reasonably-priced* cloud providers like DigitalOcean (or linode, vultr,...) with also very easy to use platforms.
* of course still pricier than dedicated hardware/VPS, however the premium for hourly billing and infrastructure maintenance is reasonable
Why would you need AWS or Cloudflare to serve that ?
I use a bunch of “freemium” services like S3 and Google Maps API and I’ve never paid a penny. I use them because they don’t cost a penny for my very limited usage, but I’m not looking forward to the day I mistakenly and disastrously exceed their free tier.
Everything can have side effects in the cloud. You can set up a cheap EC2 type T feet, and without managing your cpu usage, be charged a fair amount in unlimited burst credits (which is the default for terraform for instance).
You can quickly setup a WordPress instance with cloudfront and a invalidation Plug-in and be charged 6000 USD unadvertedly (https://wordpress.org/support/topic/amazon-cloudfront-invali...)
You can set up lambda triggers and quickly do a proof of concept for an app, but forget to correctly dimension your mem usage and be charged more than you need.
Cloud requires careful policy and topology consideration. There are many simple blocks that forms a complex mesh with opaque observability of potential vulnerabilities in both access and billing. Cloud is nice but it requires time and care. And with the shared responsibility model, you are responsible for that.
Do any other (combat) veterans smell something wrong with an Air Force Tech Controller (3C2X1) making statements like ”like back in the old days, when something would go bang or boom, and I’d run towards it” in a civilian venue? You know exactly what I mean, and we see it all the time.
If you aren’t a veteran, especially with a job even remotely related to “running towards things that go boom” please just give us some space on this one. Thanks.
1. The big cloud providers charge enormously for outgoing bandwidth. Most of us know this, but unfortunately it bites people a lot. 2. If you host big files on these clouds with no limits or warnings, it's just a matter of time before this happens to you.
This is why I don't run hobby things on these clouds. Any hobby project may have backends and services running on them, but NEVER anything user-accessible such a webserver, S3/GCS bucket, or similar. It's just too much of a "click here to bankrupt me".
For a business it's a different matter. You are making money, and you're spending money to do so. You still need to have a DDoS plan for your outgoing traffic, but it's much easier to solve these problems if you have revenue. Revenue buys time and people.
On a different note, Recently I was looking to learn AWS concepts through online courses. After so much of research I finally found this e-book on Gumroad which is written by Daniel Vassallo who has worked in AWS team for 10+ years. I found this e-book very helpful as a beginner.
This book covers most of the topics that you need to learn to get started:
If someone is interested, here's the link: https://gumroad.com/a/238777459/MsVlG
I highly recommend buying this e-book if you think AWS documentation is overwhelming.
Of course it is not ideal for companies who need their services be available for all cost, but for home users it's a nice guarantee.
- AWS CloudWatch: expensive service, virtually unusable, hard to turn it off.
- AWS overall: finding and cleaning up resources is messy. The order of creating & cleanup is not same. Closing an account is a painful process. GCP Project structure is way easier.
- AWS EKS: You create a cluster, then a node group. Deleting a cluster fails if there is a node group. You go ahead to delete a node group, it complains because of "dependencies". While you're randomly looking for a "dependency" the $ clock is still ticking. You should delete the network interface before you could delete the node group, and only then the cluster. This does not sense because if the network interface was created implicitly by the node group, i should not be responsible for deleting the network interface. There should be a symmetry in create/delete operations.
- GCP GKE: You create a cluster, then delete it. Cluster gets deleted - kudos, usability much better then with AWS EKS. But it turns out lots of LoadBalancers and Firewall rules are left over and still appear on the cloud bill. Those are implicitly created and should be cleaned up implicitly by GKE.
I want them to - I do not care if my site is offline vs. having to pay a huge bill. That should be a choice.
So I moved away from AWS. It is crazy that companies agree to such a racket (not the pricing - but the fact that you cannot set a limit).
I considered to use a virtual card with a limit on it - they could not grab more than the limit and just sue me across the pound or remove my account. But I refuse to play these games with a company who does not give a shit about billing.
A good alternative to this ever-present risk is to use a dedicated virtual private server that is unmetered. This would make mistakes like this (and yes, it is a mistake - it is his fault he didn't read the cloudfare details and publicly served a large VM image) impossible.
Here is my referral code for the one I use[1]:
https://crm.vpscheap.net/aff.php?aff=15
This also (especially) applies to startups that might suddenly take off at any moment (but don't expect to.) AWS is a ticking time bomb of unexpected charges. You never know what the Internet will bring you. Go for an unmetered VPS and have 1 single well-defined charge that doesn't change. That's what I do on my side projects.
[1] I previously asked Dan, the moderator here, if I can share in this way and he said it's okay. I don't have other affiliation with that company and have found it good. The last time I posted this I got 80 visitors and no complaints (and got upvotes), so I figure it is a good resource for people.
I've checked the traffic, it was 2.3k users for entire June, like 75 user per day at average. It is effectively nothing, why author thinks it's okay to pay 1 cent per user per month to hosting provider? $5/mo VPS can handle two orders of magnitude more.
External traffic is effectively unlimited, and a number of possible reasons (popularity, misconfigured script pulling something in a loop, someone intentionally generating traffic to hurt me) have the possibility to throw me into arbitrary amounts of debt, with the only recourse being hope that the cloud provider will be merciful.
Even if I have alerts set up: someone pulling 10 Gbit/s can generate over 100 TB per day, at $80-100 per TB. If I don't check my e-mails for weekend, I can be $30k in the hole before I notice.
This is another bad aspect of these stories.
Nice pricing AWS.
Glad I’m not the only one confused by this.
