undefined | Better HN

0 pointsfranga20003y ago0 comments

The main problems with software supply chain security are that developers using libraries don't read their code and that that code can be changed later by the author. Neither of those are problems with "AI"-generated code - it lives right in your source files and you have to be actively avoiding reading it to miss critical issues that you're familiar with.

0 comments

8 comments · 3 top-level

sanderjd3y ago· 2 in thread

Play this out to the end: instead of having dependencies, you have a giant blob of machine written code that nobody understands. This is the same problem, just with different attack vectors. Instead of trying to get a vulnerability into a popular package, attackers will try to get them into the output from common prompts.

In both cases the problem is the same, and hearkens back to Reflections on Trusting Trust. The total amount of code necessary to implement a useful system is far too large for anyone to fully understand and audit.

dinvlad3y ago

This is so unfortunate though. It really means that repetition/reimplementation (without reuse) is really the only guaranteed way of dealing with supply chain security. Other techniques like sandboxing could be useful, but are not a panacea in this case.

sanderjd3y ago

I think it's just like the physical supply chain. Everyone will pick some point on the continuum between vulnerability and reimplementation based on their individual needs.

But I think it should be clear that "well we had a black box AI make them" is not going to be a satisfying answer for militaries trying to remove hostile powers from their electronics supply chains. No different with software.

1 more reply

pjc503y ago· 2 in thread

> have to be actively avoiding reading it

A question of volume, surely? It might be OK for snippets, but once you've added a million lines of AI code to your codebase, when are you going to get around to reading it?

Like libraries, you hit a button and add thousands to millions of LOC. If it appears to work, are you going to read it?

sanderjd3y ago

Reading the AI code and understanding it would be a harder task than writing it yourself, in my opinion. And that will only be more true over time if it becomes more common to have prompt writing skills than code writing skills.

dinvlad3y ago

I’d even argue the next logical step would be FPGA-type programming where there isn’t really a human-readable program anymore, but just a model loaded into hardware.

1 more reply

Jeff_Brown3y ago· 1 in thread

Algebraic effects could help a lot. They let you quickly discard giant swaths of pure code, or even impure code that's surely harmless because its effects are limited to, say, graphics or audio output, and focus on the operations that use sensitive things like the file store, the internet, etc.

blurbleblurble3y ago

Yes! I'm looking forward to trying these generative code tools with algebraic effects and similar.

j / k navigate · click thread line to collapse

0 comments

8 comments · 3 top-level

sanderjd3y ago· 2 in thread

dinvlad3y ago

sanderjd3y ago

I think it's just like the physical supply chain. Everyone will pick some point on the continuum between vulnerability and reimplementation based on their individual needs.

1 more reply

pjc503y ago· 2 in thread

> have to be actively avoiding reading it

A question of volume, surely? It might be OK for snippets, but once you've added a million lines of AI code to your codebase, when are you going to get around to reading it?

Like libraries, you hit a button and add thousands to millions of LOC. If it appears to work, are you going to read it?

sanderjd3y ago

dinvlad3y ago

I’d even argue the next logical step would be FPGA-type programming where there isn’t really a human-readable program anymore, but just a model loaded into hardware.

1 more reply

Jeff_Brown3y ago· 1 in thread

blurbleblurble3y ago

Yes! I'm looking forward to trying these generative code tools with algebraic effects and similar.

j / k navigate · click thread line to collapse