Datadog is good, sentry too, but after running a cloud practice for a major world business, I prefer to have my sensitive system logs and traces in house.
> we lost visibility into production systems that depend fundamentally on continuous observability signals to operate safely.
The Datadog message implies that Deductive wasn't paying for any service from Datadog: "We've noticed you're actively evaluating Datadog" and "our Master Subscription Agreement that you accepted by using our service".
And Deductive apparently did this from Feb to Dec 2025. Quite a long time for a free evaluation, but perhaps they were just using the very limited free tier?
It's a little strange to be relying on a free tier or evaluation for "production systems that depend fundamentally on continuous observability". Presumably it couldn't have been that important to Deductive, otherwise they would have paid for the service they were "depending fundamentally" on.
This paragraph from the article makes it clear otherwise.
The blog says they were paying(?)
It's a "*.ai" company. Deductive probably spent more human time on their fancy animated landing page than engineering their actual system. If they vibe coded most of their product, I wouldn't be surprised if they didn't even know they were using Datadog until they got the email.
We have been using few different SRE agents and they all fucking suck. The way they are promoted and run always makes them eager to “please” by inventing processes, services, and work-arounds that don’t exist or make no sense. Giving examples will always sound pity or “dumb”. Every time I have to explain to management where SRE agent failed they just hand wave it and assume it’s a small problem. And the problem is, I totally get it. When the SRE agent says “DNS propagation issues are common. I recommend flushing dns cache or trying again later” or “The edge proxy held a bad cache entry. Cache will eventually get purged and the issue should be solved eventually” sounds so reasonable and “smart”. The issue was in DNS or in the proxy configuration. How smart was the SRE agent to get there? They think it’s phenomenal and it may be. But I know that the “DNS issue” isn’t gonna resolve itself because we have a bug in how we update DNS. I know the edge proxy cache issue is always gonna cause a particular use case to fail because the way cache invalidation is implemented has a bug. Everyone loves deflection (including me) and “self correcting” systems. But it just means that a certain class of bugs will forever be “fine” and maybe that’s fine. I don’t know anymore.
So in some sense the agent is doing a pretty good job…
That experience made me think we're getting close to SRE agents being a thing.
And as the LLM makers like to reiterate, the underlying models will get better.
Which is to say, I think everyone should have some humility here because how useful the systems end up being is very uncertain. This of course applies just as much to execs who are ingesting the AI hype too.
That said, I think you're right that you can't really replace an Operations staff, as there will always need to be a human making complex, multi-dimensional decisions around constantly changing scenarios, in order to keep a business operational.
We built 100% in-house pieces for all of this at a major fintech a decade ago. Everything worked and single teams could manage these systems.
Someone in leadership said we had to get rid of all "weirdware". Open solutions weren't robust, so we went commerical.
SignalFX got acquired, immediately 10x'd our prices and put all hands on deck to migrate. Unscheduled, stressful, bullshit. We missed the migration date and had to pay anyway.
LaunchDarkly promised us the moon to replace the system my team built. It didn't work with Ruby, Go, and the Java client sucked. It couldn't sync online changes at runtime like our five nines distributed and fault tolerant system could. We had to upstream a ton of code. And their system still sucked by the time I left the project.
These systems need to be open and owned by us. Managed is okay, but they shouldn't be proprietary offerings.
I could extend that one step further to cloud itself, but that's an argument for another day.
Absolutely. OSS platforms like k8s got a long way. Openstack was the dream (deeply flawed in execution). If we want to seriously talk about resilience we can’t accept that almost all major clouds run proprietary systems and we just have to trust them that they’ll be around forever.
> These systems need to be open and owned by us. Managed is okay, but they shouldn't be proprietary offerings.
You could say this about all software in the world, but good luck with that... people who make money off of making things and selling things are going to keep doing so in non-open ways, because it's advantageous. And customers will keep buying them, because it's better than the alternative.
Control plane and observability are key concerns of a fintech handling billions in daily transaction volume.
We had teams building and managing our solutions. After the migrations, we had teams managing the integrations. The headcount didn't change, we just wound up paying external vendors and sequencing multiple provider moves and company wide migrations. The changes caused several outages and shifted OKRs.
At the end of the day, you just need to make good decisions based on honest analysis of your needs, capabilities, and general context.
Why does the gate say “blocked” if the stuff is clearly still flowing through it? Having no image is better than shitty ones.
Hype Phrase! (but like super over the top) "That world no longer exists." "code is cheap" "That future is already here."
repeat.
Very ai. Ai loves to summerise stuff in cheesy little bits like that. Very linkedin. Very bad writing too.
But looking back at it with an AI nose going, it does have a ton of AI slop feeling to it. LinkedIn slop is kind of interesting to read. The repetition is pretty obvious, though. This reads like it was written by a 10th grade english class trying to fit a very specific structure. Like every section had to check a list of requirements, which it did cuz it's AI.
- AI mentioned in the subtitle
- AI generated image
Straight to /dev/null with this slop.
