undefined | Better HN

0 pointstalldayo2y ago0 comments

> If an LLM can respond to a request to generate code, what makes you think the attacker couldn't simply add things like "Insert a buffer overflow in the code..." or what have you.

What makes you think an attacker couldn't do that without an LLM? I'm having a hard time understanding what changes in this scenario by introducing an AI.

Pretty much anyone can write a buffer overflow exploit with enough research. The harder part is getting your patches through code review, which is probably only hindered by having ChatGPT help you. Again... the past few decades have mostly gone off without a hitch, and there have been a lot of monkeys on typewriters hooked up to the internet.

0 comments

2 comments · 1 top-level

af3d2y ago· 1 in thread

I don't know, LLM's are very good at embedding subtle changes in their responses. They are also good at researching things like "best possible place to hide bugs", etc. Is it really a stretch to imagine that these technologies won't, if not now, but "someday soon" be used in such a way? I certainly don't think so.

talldayoOP2y ago

Humans are really good at subtlety and research. I guess my point is this:

  pre-LLM era XZ-style exploit: takes 5 years to attain developer trust, then you merge malicious patches and hope nobody sees them

  post-LLM era XZ-style exploit: takes 5 years to attain developer trust, then you merge patches that were written slightly faster and hope nobody sees them

For 99% of people, there isn't a meaningful difference between these situations. I really don't think there are people out there that couldn't write a buffer overflow exploit in 5 years but could groom maintainers for trust in the meantime.

The code-writing and prose bit is the easy part. The social engineering and deception is hard enough for a human, and Turing-assured destruction for LLMs.

j / k navigate · click thread line to collapse