Seriously, if you’re at the point that you’re doing sophisticated analysis of cloud costs, consider dropping the cloud.
I swear the people who say go on premise have no idea how much the salary costs of someone who will not treat their datacenter like a home lab is. Even Apple iCloud is in AWS and GCP because of how economical it is, you suck at the cloud you think you have to go back on prem, or you just don't give a shit about reliability (start pricing up DDoS protection costs at anything higher than 10G and tell me the cloud is more expensive).
We spend 100k+ on AWS bandwidth and it's still cheaper than our DIA circuits because we don't have to pay network engineers to manage 3 different AZs.
Some people act like it's some kind of black magic. It's not. We've some customers in our DC and some on AWS for various reasons. AWS isn't less problematic. AWS is about 10x more expensive. Both on prem and cloud require people familiar with them and cloud-engineers are in no way cheaper.
Only meaningful problem is that on-prem requires some up front cost&time. That can be mitigated by leasing and other means, but indeed can be an issue for small businesses.
There is too much nuance to say one is better than the other. In some cases using a IaaS is more economical, in other cases it's not.
For Apple, the same is also true[0] to say "Even Apple is running their own datacenters because of how economical it is"
0 - https://dgtlinfra.com/apple-data-center-locations/#:~:text=a....
What does this mean? They steal company resources for themselves, or just configure things incompetently?
I haven’t yet meet anyone at a company that heavily uses cloud and still doesn’t have the same number of salaried infra people as on-prem.
Active-active, five nines, fault tolerance. Hard stuff. But managing on-prem is no harder.
This is what we're paid for.
Which would mean that you loose part of the reason to use the cloud in the first place... A lot of org move to cloud based hosting because it enable them to go way further in FinOps / cost control (amongst many other thing).
This can make a lot of sense depending on your infra, if you have some fluctuation in you needs (storage, compute, etc...), cloud based solution can be a great fit.
At the end of the day, it is just a tool. I worked in places where I SSH´d into the prod bare-metal server to update our software, manage the firewall, check the storage, ... and all that manually. And I worked in places where we were using a cloud provider for most of our hosting needs. I also handle the transition from one to the other. All I can say is: It's a tool. "The cloud" is no better or worse than a bare-metal server or a VPS. It really depends on your use-case. You should just do your due diligence and evaluate the reason why one would fit you more than the other, and reevaluate it from time to time depending on the changes in your environment.
This whole "cloud bad" is just childish.
I think a lot of orgs move to cloud simply because it's popular and gartner told them so.
But taking a step away from that, it's really about self-service. When the alternative is logging a ticket for someone to manually misconfigure a VM and then fail to send you the login credentials, then your delivery is slow.
When you're chasing revenue, going slow means you're leaving money on the table. When you're a big bureaucratic org, it means your middle managers can't claim to have delivered a whole bunch of shit. Nobody likes being held up, but that's what infrastructure teams historically do.
Not childish…. it’s a growing line of thought in the IT community that has bought the cloud sell unquestioningly for 20 years.
AWS is Kafkaesque though
The blog post's solution is relatively simple to put in place ; if you're already locked-in to AWS, dropping it will cost quite a lot and this might be a great middle ground in some cases.
If you're actually using the features of cloud - i.e. managed services - then this involves building out an IT department with skills that many companies just don't have. And the cost of that will easily negate or exceed any savings, in many cases.
That's a big reason that the choice of cloud became such a no brainer for companies.
Now as to why the AWS pricing the way it is... we may only guess, but likely to promote one service over the other.
But there is no way that cross-AZ traffic costs AWS anywhere near what they charge for it.
Cloud bandwidth pricing is… well the best analogies are very inappropriate for this site.
tilaa.com
vultr.com
hetzner.com
linode.com
That's just not gonna happen for a lot of services.
So what do I do if I'm at the point where I'm doing sophisticated analysis of on-prem costs? Do I move to the cloud?
then load the data into the datacentre and then just pay for the last sync / delta.
Lightsail instances can be used to "proxy" data from other AWS resources (eg EC2 instances or S3 buckets). Each Lightsail instance has a certain amount of data transfer included in it's price ($3.5 instance has 1TB, $5 instance has 2TB, $10 instance has 3TB, $20 instance has 4TB, $40 instance has 5TB). The best value (dollar per transferred data) is the $10 instance, which gives you 3TB of traffic.
Using the data provided by the post:
3TB worth of traffic from an EC2 would cost $276.48 (us-east-1). 3TB worth of traffic from a S3 bucket would cost $69.
Note: one downside of using Lightsail instances is that both ingress and egress traffic counts as "traffic".
> 51.3. You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services (e.g., proxying network traffic from Services to the public internet or other destinations or excessive data processing through load balancing or content delivery network (CDN) Services as described in the technical documentation), and if you do, we may throttle or suspend your data services or suspend your account.
This requires proving the users intent, which is not obvious except in the most blatant of cases (i.e. using Lightsail as a bent-pipe by writing the exact bytes you're reading). If it is a "CSV to Parquet translation layer", how would AWS possibly prove it's anything other than what it claims to be? You'd be paying a few more cents for compute, but that's the price of plausible deniability
You can download 1TB of data for free from AWS each month, as Cloudfront has a free tier [1] with 1TB monthly egress included. Point it to S3 or whatever HTTP server you want and voila.
[1] It used to be 50GB per month for the first 12 months. It was changed to 1TB free forever shortly after Cloudflare posted https://blog.cloudflare.com/aws-egregious-egress
Nitpick: $5 for 2TB is better than $10 for 3TB.
1. Ask for discounts if you are a big AWS customer (e.g., spend $1mln+/year). At some point, they were huge for inter-AZ transfers.
2. Put things in one AZ. Running DB in "b" zone and your only server in "a" is even worse than just standardizing on one zone.
3. When using multiple AZ do load aware AZ balancing.
There must be use cases for this, but I lack imagination. Cost? But not cost?
Many companies run servers without considering AZ. Then you can get the "best" of the worlds:
1. Your service is down if either of AZs gets hiccups.
2. You pay network charges and latency cost.
https://docs.aws.amazon.com/ram/latest/userguide/working-wit...
But in the context of data transfer costs, this would actually increase the costs, because there's a small surcharge for Intelligent Tiering - and the only relevant storage class for sidestepping data transfer costs is standard storage (because it's the only one with free download), so Intelligent Tiering won't provide value.
https://discourse.nixos.org/t/the-nixos-foundations-call-to-...
[1]: https://cloud.google.com/storage/pricing-announce#network
It works, but i have no use case for it. 100TB egress to the internet costs about 7000$ ... i think, i could do it for 20$-50$.
Also, the prices he quotes are label prices, if you are a customer and you pre purchase your bandwidth under an agreement, it gets _significantly_ less expensive.
But yes, the moment you actually produce significant amounts of egress traffic it gets absurdly expensive. And I would expect competitors like R2 to gain ground if they can provide reasonably competitive reliability and performance.
Maybe raising the cost of transient storage? e.g. If you have to pay for a minimum of a day's storage - but even if that was the case this would still be cost-effective, and at any rate it seems very unnatural for AWS to charge on such granularity.
+ I would guess that S3 is orders of magnitude more profitable for AWS than cross-AZ charges, so I'm not sure they'd consider it a loss-leader.
It would definitely piss a lot of people off as it is adding to their bill, but it could likely be done in a way that makes exploiting this for just data transfer not worth it without adding huge costs to most "real" use cases.
What's more odd now that 1gigabit+ home connections are available, it should be obvious to anyone doing the math that it can't cost that much, otherwise a 200GB CoD install would be costing the ISP $20.
Some how AWS has to rip you off so if there is a non rip off gateway to the ripoff then if you can use the non rip off to avoid another rip off, they will close the “loophole”.
If too many people do this - AWS will "just close the loophole".
There's not one AWS - there are probably dozens if not hundreds of AWS - each with their own KPIs. One wants to reduce your spend - but not tell you how to really reduce your spend.
If you make something complex enough (AWS) - it will be impossible for customers to optimize in any one factor - as everything is complected together.
Another example is using CloudFront. AWS wants you to use CloudFront, so they make CloudFront cheaper than other types of data egress.
https://www.lastweekinaws.com/blog/aws-cross-az-data-transfe...
I remember a bizarre situation where the AWS sales guys wouldn't budge on bandwidth pricing even though the company wouldn't be viable at the standard prices.
I'm back to managing my own systems. So much cheaper and less chance of nonlinear bills.
We've built https://nodeshift.com/ with the idea that cloud is affordable by default without any additional optimization, you focus on your app with no concerns on costs or anything else.
Treat this as an interesting hacking exercise, but do not deploy a solution like this to production (or at least get your account managers blessing first), lest you risk waking up to a terminated AWS account.
It’s also possible their billing system can’t detect transient storage usage. Request billing would work differently from how billed storage is tracked. It depends on how billing is implemented but would be my guess. That may change in the future.
Suppose you store the data there for 6 minutes. Then there's an 90% probability that the sampler misses it entirely and you pay $0. But there's a 10% probability that the sampler does catch it. Then you pay for a whole hour even though you used a fraction of that.
Over many events, it averages out close to actual usage[1]. In 9 out of 10 cases, you pay 0X actual usage. In 1 out of 10 cases, you pay 10X actual usage. (But you can't complain because you did agree to 1-hour increments.)
---
[1] Assuming no correlation between your timing and the sampler's timing. If you can evade the sampler by guessing when it runs and carefully timing your access, then you can save a few pennies at the risk of a ban.
> Moreover, uploading to S3 - in any storage class - is also free!
Depending upon how much data you’re transferring in terms of storage class, number of API calls your software makes to do so, and the capacity used, you may incur charges. This is very easy to inadvertently do when uploading large volumes of archival data directly to the S3 Glacier tiers. You absolutely will pay if you end up using millions of API calls to upload tens of millions of objects comprising tens of terabytes or more.
I updated the phrasing.
If all services, then things like whole or most of a single AZ being borked happens fairly often.
However sounds like it was deleted almost right away. In that case the charge might be 0.03 / 60 if the data was around for a minute. Normally I would expect AWS to round this up to $0.01..
The TimedByteStorage value from the cost and usage report would be the ultimate determinant here.
I’ve on three or four occasions the last three months notices that I got served a completely different SSL certificate than the domain I was visiting, of a domain that often could not be reached on publicly - probably pointing to some organizations internal OTA environment. In all occasions the URL I wanted to visit and the DNS of the site I was then actually visiting were located in AWS. Then less than a minute elapsed the issue is resolved.
I first thought it must be my side, my DNS server malfunctioning or something, but the served sites could not be accessed publicly anyway, and I had the issue on two separate networks with two separate clients and separate DNS servers. I’ve had it with polar.co internal environment, bank of ireland (iirc), multiple times with download.mozilla.org and a few other occassions.
I contacted AWS on Twitter about it, but just got some generic pointless response I should make an incident - but I’m just some random user, I’m not an AWS client. Somehow I could not get it clear to the AWS support on the other side of Twitter.
Do you setup the same infrastructure in two or more VPS instances and then load balance? (say, [1]). Feels a bit of an ... artisanal solution, compared to using something like AWS ECS.
