undefined | Better HN

0 pointsp1necone1mo ago0 comments

The "this test failure is preexisting so I'm going to ignore it" thing has been happening a lot for me lately, it's so annoying. Unless it makes a change and then immediately runs tests and it's obvious from the name/contents that the failing test is directly related to the change that was made it will ignore it and not try to fix.

0 comments

Shebanator1mo ago

This problem has been around for a long time. Not only that but it would say this even when the problems were directly caused by their code.

I put a line in my CLAUDE.md that says "If a test doesn't pass, fix it regardless of whether it was pre-existing or in a different part of the code."

latentsea1mo ago

This should be part of the system prompt. It's absolutely unacceptable to just to not at least try to investigate failures like this. I absolutely hate when it reaches this conclusion on its own and just continues on as if it's doing valid work.

foltik1mo ago

Based on the recent leaks, their system prompt explicitly nudges the model not to do anything outside of what was asked. That could very well explain why it’s not fixing preexisting broken tests.

“Don't add features, refactor code, or make "improvements" beyond what was asked.”

https://www.dbreunig.com/2026/04/04/how-claude-code-builds-a...

hakanderyal1mo ago

And it's very valid. Because otherwise you would ask Claude to trim a tree and it would go raze the whole forest and plant new seeds. This was the primary pain point last year, especially with Sonnet.

1 more reply

flakes1mo ago

> "this test failure is preexisting so I'm going to ignore it"

Critical finding! You spotted the smoking gun!

cmrdporcupine1mo ago

I will note that this "out" that Claude takes was a) less frequent in Opus 4.5 and that time frame and b) notably not something that Codex does.

I don't trust the code that Claude writes at all, if I have to use it (they gave me a free month recently, so I use it...) I not only review it carefully but have Codex do a thorough review.

Claude "cheats" and leaves hacks and has Dunning-Kruger.

All of this is very exhausting. I am enjoying writing my own code with these tools (to get long running personal projects out the door) but the effect that these tools are having on teams is terrifyingly corrosive and it's making me want to take an early retirement from the profession.

Yes we can write a lot of code quickly. But at what cost? And what even use is all this code now anyways?

dboreham1mo ago

That said I've worked with several humans who did/said the exact same thing.

boesboes1mo ago

But did they say that about tests they just added themselves too? Had claude try that on me a couple of times >_<

gmassman1mo ago

Usually these were the developers who said their code didn’t need tests because it’s obviously correct/too simple to need them. And then their bug causes a crash that needs to be fixed over the weekend :/

j / k navigate · click thread line to collapse