Show HN: Infracost (YC W21): Be proactive with your cloud costs

Great point about the multiple API calls. One of the big problems we’ve heard about using SCPs is that they are too late. If a deployment fails because of them the developer needs to go through another pull request/code review.

Estimating costs for serverless technologies upfront is definitely challenging. We're thinking of bringing in the last 30 days of usage for these resources to give engineers some visibility.

I've not used the product, so it may already do this, but does it ask you for the data it needs in the Pull Request?

I have experience interacting with a logging system, where any diff to the logged data would need a tag like `log_size_increase=3 bytes` – the CICD system would then turn this, with the data already available, into an estimate of the overall extra storage needed.

Perhaps the same could be done. Rather than figuring out "usage" of some serverless systems, which is a very vague question and therefore hard to answer, perhaps it could be more specific. For example, how many requests per second is it expected to receive? Or, which other serverless functions call it (and therefore which will it necessarily have the scale of). Or, what increase in usage would be expected for this change.

> but in practice this is a big loser in AWS where a staggering number of resources are created by one API call and then tagged as a followup API call

We have a bot at work that sends you (or a DL with a bunch of people) a nastygram if you forget to tag your resources, but it doesn't know this. So if CloudFormation isn't done, you'll get the email and then have to respond to everyone with a screenshot showing that you didn't in fact goof it up. I wonder if you can make it so EventBridge (or however it's implemented, I'm not sure) delay an event for 30 seconds so they don't actually look until CF is done tagging.

Great point - indeed FinOps teams consistently rank "empowering engineers to take action" as their number 1 challenge (https://data.finops.org) - and by that they mean the human and organization dynamics of the culture change they want to create across the org.

The testing analogy is a good one as this feature also shows the engineers the current "failing policies" on the main branch too, so whilst they could merge the pull request without fixing the tagging issue, it'll just get added to the list. And maybe like tests, they group them into one task and go through to fix them all every so often to get the main branch back to green!

For anyone following at home, once you've identified a test as flaky, your next action should be to turn it off. Nothing good comes from keeping flaky tests around. Detect them as soon as you can and either fix them _right there_ or skip them.

I've used this in practice in a company of ~80 developers at the time, applied it because read about it in some Dropbox papers, and have since seen it work in 2 other companies. Skip your flaky tests!!

Counterpoint to this is the rate at which you can grab some new infra and try something out is amazing. You need a Redis cache? Or a GPU? Here have one in a few seconds, instead if next month or never.

It is a bit like complaining about making tractor engines more efficient when you didn’t have those problems using animals and ploughs.

How else will people make money (if they are a startup) or get promotions (if they are in a corporate).

Bingo - so Infracost will tell you before you launch anything how much it'll cost. Now scale that to a few thousand developers across a large company, and it's very impactful.

Backfilling tags works, but the issue is if Terraform isn't updated, it causes drift - it's much better to fix it at the root, so that's what Infracost helps with

Awesome, yeah we’ve seen people using this method and the main complaint we’ve heard is this is annoying for developers since it blocks their deployments when they run `terraform apply`, so they need to create new pull requests and wait for another code review. Combining both can definitely help with this.

Not sure if with shift-left you mean specifically shifting left infracost and FinOps or general Infrastructure-as-Code shift left.

In case it's the latter, I can recommend Spacelift[0] - a specialized CI/CD tool for IaC and supports all the tools you've mentioned. It basically helps you build policies and orchestrate your infra (don't want to go into too much detail in this comment) to scale it to bigger teams and setups. Policies to enforce tagging would indeed be a good example.

It integrates with infracost too, but obviously just for the tools infracost works with, no CloudFormation.

Disclaimer: Work at Spacelift so obviously take the recommendation with a grain of salt, but I do legitimately think it's a great tool.

[0]: https://spacelift.io

P.S. Congrats on the Show HN Infracost team!

Yep, for sure! It's on the roadmap. We are friends with the folks at Pulumi too. Love what they are building, so hopefully we will get some bandwidth and add support. And Cloud Formation too. Azure ARM ... haha there is a lot more to build :)

Hi, I think the key issue with both the Azure policy, and the Amazon services is that they only work after a pull request has been merged. Then the build fails, and the engineer has to come back to their code, make a new Pull Request and then send it again, till it passes.

That's the feedback we got from users, so with Infracost, the Pull Request itself tells the engineer what needs to be done, along with exact code line numbers etc before going any further, so everything is fixed within the same pull request. Also, it works across all cloud providers, so FinOps can set central tags in a uniform manner no matter where the engineers are launching resources.

Interesting idea! The pull request authors are shown out of the box but we hadn't thought of using git to find the user for each resource on the main branch. Most organizations end-up tagging the resources with some sort of owner or team so they can group the costs using that and track it per team/service/product over time. That's often how FinOps teams start to create a sense of ownership for cloud costs amongst teams.

Yes, usage metrics are set via a configuration file, which you can also check into git. Changes to resources as well as usage estimates contribute to the forecasted costs

Vantage is awesome - I've talked with Ben (their CEO) a few times. There are a lot of tools that start from the cloud bills, and give you visibility of everything (Vantage, Cloudability, Cloud Health Tech, Flexera etc) - all of these tools are reactive in nature as they start from the cloud bills. Infracost sits where your code sits, and therefore it can be proactive; before anything is launched and costs money, it'll tell you how much it is going to cost. So if you have a budget of $1K, and you try to launch a 24xl instance, it'll tell you that you budget will be blown, before you've launched the resource. Making it all proactive.

Thanks! Yep, we're partners with HashiCorp and worked on that integration early on :)

hahaha love it - it'll keep evolving, and get more interesting for sure. I just think back to Dev and Ops, and how it went from a culture to a job title; testing was another one.