I’m saying they are not black boxes at all. Not just “technically”
> The point is that there is no one implementation of C. And the same is true for any other language too (to a lesser extent).
That doesn’t matter for my argument.
> It doesn't matter whether they are black boxes if you don't routinely inspect the boxes. Perhaps you do, but for most people they don't. So it doesn't matter as much.
It does matter. That’s my whole point.
For two main reasons:
First, you have the option to understand how a compiler translates C to assembly. You do NOT have that option with LLMs
With a compiler, you are abstracting the underlying machine code, but you’re not hiding it entirely.
Secondly, with an LLM there’s no tractable connection between your prompt and your code. That’s a huge issue when it comes to understanding the resulting program.
Imagine a world where LLMs turn raw language into assembly (or, more realistically, LLVM IR)
How would you reason about your resulting program?
You could give the same prompt to the same LLM and get wildly different resulting code, since LLMs are nondeterministic.
How would you debug? How would you ensure that small changes to your LLM prompt doesn’t change parts of your resulting code that worked well?
How would you even understand what your program is doing?
You couldn’t do any of those things, because there’s no deterministic relationship between the prompt and the resulting program.
