I'm not just talking about the user

Its not like anthropic can just set a breakpoint in the model and debug

I have been able to build plenty of stuff with a pretty plain emacs + ghci for years...neither are black boxes. Except maybe my brain driving them.

not really. Most of the technology is not black box but something of a grey box. You usually choose to treat it as a black box because you want to focus on your problems/your customers but you can always focus on underlying technologies and improve them. Eg postgresql for me is a black box but if I really wanted or had need I could investigate how it works.

Those black boxes are usually deterministic.

Yeah someone should’ve told that to Donald (Knuth)

/s

For those who don’t know, Knuth implemented the typesetting system TeX just to make sure his book’s typesetting was correct.

You can pretty much only innovate when you reject the blackbox and decide to make a better one.

Otherwise you’re likely implementing something you could probably get off-the-shelf, which is ok, but also something that you could just… not implement.