Yikes.
Probabilistic mitigations work against probabilistic attacks, I guess - but exploit writers aren't random, they are directed, and they find the weaknesses.
"This is true by definition as the QuickJS vulnerability was previously unknown until I found it (or, more correctly: my Opus 4.5 vulnerability discovery agent found it)."
Lots of vulnerabilites get stopped dead by these mitigations. You almost always need multiple vulnerabilities tied together, which relies on a level of vulnerability density that's tractable. This is not just busywork.
WASM adds a layer, but the first thing anyone will do is look for a way to escape it. And unless all software faults and hardware faults magically disappear, it'll still be a constant source of bugs.
Pitching a sandbox against ingenuity will always fail at some point, there is no panacea.
[0] https://instatunnel.substack.com/p/the-wasm-breach-escaping-...
I hope not, my laptop is slow enough as it is.
> Leak a libc Pointer via Use-After-Free. The exploit uses the vulnerability to leak a pointer to libc.
I doubt Rust would save you here unless the binary has very limited calls to libc, but would be much harder for a UaF to happen in Rust code.
Combine that with a minimal docker container and you don't even need a shell or anything but the kernel in those images.
(I’m not trying to be facetious or troll or whatever. Stuff like this is what motivated me to do it.)
Wouldn't GP's approach work with any other executable using libc? Python, Node, Rust, etc?
I fail to see what is specific to either C or QuickJS in the GP's approach.
> Yikes.
Yep.
I actually think this result is a little disappointing but I largely chalk it up to the limited budget the author invested. In the CTF space we’re definitely seeing this more and more as models effectively “oneshot” typical pwn tasks that were significant effort to do by hand before. I feel like the pieces to do these are vaguely present in training data and the real constraints have been how fiddly and annoying they are to set up. An LLM is going to be well suited at this.
More interestingly, though, I suspect we will actually see software at least briefly get more secure as a result of this: I think a lot of incomplete implementations of mitigations are going to fall soon and (humans, for now) will be forced to keep up and patch them properly. This will drive investment in formal modeling of exploits, which is currently a very immature field.
Can you elaborate more on this with pointers to some resources?
The defensive side needs everything to go right, all the time. The offensive side only needs something to go wrong once.
It's a subtle difference from what you said in that it's not like everything has to go right in a sequence for the defensive side, defenders just have to hope they committed enough into searching such that the offensive side has a significantly lowered chance of finding solutions they did not. Both the attackers and defenders are attacking a target program and sampling the same distribution for attacks, it's just that the defender is also iterating on patching any found exploits until their budget is exhausted.
It's probably more worrying as you get script kiddies on steroids which can spawn all around with same mindset as even the dumbest significant geopolitical actor out there.
I don't think so. From a pure mathematical standpoint, you'd need better (or equal) results at avg@1 or maj@x, while the attacker needs just pass@x to succeed. That is, the red agent needs to work just once, while the blue agent needs to work all the time. Current agents are much better (20-30%) at pass@x than maj@x.
In real life that's why you sometimes see titles like "teenager hacks into multi-billion dollar company and installs crypto malware".
I do think that you're right in that we'll see improved security stance by using red v. blue agents "in a loop". But I also think that red has a mathematical advantage here.
> I don't think so. From a pure mathematical standpoint, you'd need better (or equal) results at avg@1 or maj@x, while the attacker needs just pass@x to succeed.
Executing remote code is a choice not some sort of force of nature.
Timesharing systems are inherently not safe and way too much effort is put into claiming the stone from Sisyphus.
SaaS and complex centralized software need to go and that is way over due.
https://projectzero.google/2024/10/from-naptime-to-big-sleep...
List of vulnerabilities found so far:
There are countless bugs to fund.
If the offender runs these tools, then any bug they find becomes a cyberweapon.
If the defender runs these tools, they will not thwart the offender unless they find and fix all of the bugs.
Any vs all is not symmetric
A) 1 cyber security employee, 1 determined attacker
B) 100 cyber security employees, 100 determined attackers
Which is better for defender?
Given the large number of unmaintained or non-recent software out there, I think being worried is the right approach.
The only guaranteed winner is the LLM companies, who get to sell tokens to both sides.
Why? The attackers can run the defending software as well. As such they can test millions of testcases, and if one breaks through the defenses they can make it go live.
I'm quite optimistic about AI ultimately making systems more secure and well protected, shifting the overall balance towards the defenders.
The reflects my experience too. Opus 4.5 is my everyday driver - I like using it. But Codex 5.2 with Extra High thinking is just a bit more powerful.
Also despite what people say, I don't believe progress in LLM performance is slowing down at all - instead we are having more trouble generating tasks that are hard enough, and the frontier tasks they are failing at or just managing are so complex that most people outside the specialized field aren't interested enough to sit through the explanation.
Gemini either does a Monet or demolishes your bathroom and builds a new tuna fishing boat there instead, and it is completely random which one you get.
It's a great model but I rarely use it because it's so random as to what you get.
The quality of output you see from any LLM system is filtered through the human who acts on those results.
A dumbass pasting LLM generated "reports" into an issue system doesn't disprove the efforts of a subject-matter expert who knows how to get good results from LLMs and has the necessary taste to only share the credible issues it helps them find.
The Industrialisation of Intrusion
By ‘industrialisation’ I mean that the ability of an organisation to complete a task will be limited by the number of tokens they can throw at that task. In order for a task to be ‘industrialised’ in this way it needs two things:
An LLM-based agent must be able to search the solution space. It must have an environment in which to operate, appropriate tools, and not require human assistance. The ability to do true ‘search’, and cover more of the solution space as more tokens are spent also requires some baseline capability from the model to process information, react to it, and make sensible decisions that move the search forward. It looks like Opus 4.5 and GPT-5.2 possess this in my experiments. It will be interesting to see how they do against a much larger space, like v8 or Firefox.
The agent must have some way to verify its solution. The verifier needs to be accurate, fast and again not involve a human.
[1] If their experiment even demonstrates what it purports to demonstrate. For anyone to give this article any credence, the exploit really needs to be independently verified that it is what they say it is and that it was achieved the way they say it was achieved.
With the CVE reports some poor maintainer has to go through and triage them, which is far more work, and very asymmetrical because the reporters can generate their spam reports in volume while each one requires detailed analysis.
For example if you just ask an LLM in a browser with no tool use to "find a vulnerability in this program", it'll likely give you something but it is very likely to be hallucinated or irrelevant.
However if you use the same LLM model via an agent, and provide it with concrete guidance on how to test its success, and the environment needed to prove that success, you are much more likely to get a good result.
It's like with Claude code, if you don't provide a test environment it will often make mistakes in the coding and tell you all is well, but if you provide a testing loop it'll iterate till it actually works.
For an interesting overview of the above theorem, see here[1].
0 - https://en.wikipedia.org/wiki/Infinite_monkey_theorem
1 - https://www.yalescientific.org/2025/04/sorry-shakespeare-why...
The AI slop you see on curl's bug bounty program[1] (mostly) comes from people who are not hackers in the first place.
In the contrary persons like the author are obviously skilled in security research and will definitely send valid bugs.
Same can be said for people in my space who do build LLM-driven exploit development. In the US Xbow hired quite some skilled researchers [2] had some promising development for instance.
[1] https://hackerone.com/curl/hacktivity [2] https://xbow.com/about
His co-founder on optimyze was Sean Heelan, the author of the OP.
Scary.
If forking is blocked, the exit handler can't do it either. If it's some variant of execve, the sandbox is preserved so we didn't gain much.
Edit: ok, I get it! Missed the "Goal: write exactly "PWNED" to /tmp/pwned". Which makes the sandbox part way less interesting as implemented. It's just saying you can't shell out to do it, but there's no sandbox breakout at any point in the exploit.
We need new platforms which provide the necessary security guardrails, verifiability, simplicity of development, succinctness of logic (high feature/code ratio)... You can't trust non-technical vibe coders with today's software tools when they can't even trust themselves.
There are other nuances which may offset the asymmetry a bit; for example the security analyst generally has much more visibility over the company's code than the hacker does.
That said, I stand by my original point because I think that building secure systems is really hard; it's much more effort per unit of functionality to build the system correctly (and doing that for every part of it) than it is to crack it (by finding a single hole).
On the side of defense, you need to understand a lot of nuance about how your system works and how parts interact to make it secure; any neglect can potentially be a critical vulnerability which can compromise the entire system.
On the side of offense, sometimes mindless prodding can uncover a critical vulnerability. The intelligence/thinking requirement is lower; it's more about knowledge than thinking.
For example, there are some special payloads which you can send which may pose a problem for different systems built by different companies because the companies share the same underlying engine or they fell victim to the same footgun. I think this aspect is much more important than my previous argument.
The interesting part is still finding new potential RCE vulnerabilities, and generally if you can demonstrate the vulnerability even without demonstrating an E2E pwn red teams and white hats will still get credit.
I think the main challenge for hackers is to find 0day vulnerabilities, not writing the actual exploit code.
>This is true by definition as the QuickJS vulnerability was previously unknown until I found it (or, more correctly: my Opus 4.5 vulnerability discovery agent found it).
1) it becomes increasingly more dangerous to dl stuff from the internet and just run it, even its opensource, given normally people don't read all of it. for weird repos I'd recomment to do automated analysis with opus 4.5 or the gpt 5.2 indeed.
2) if we assume adversaries are using LLMs to churn exploits 24/7, which we should absolutely do, perhaps the time where we turn the internet off whenever is not needed, is not far.
You are right: 30 years ago, it was safe to go to vendor XY page and download his latest version and it was more or less waterproof. Today with all these mirror sites, very often better SEO ranking than the original, its quite dangerous: In my former bank we had a colleague who installed a browser add-in that he used for years (at home and in the bank); then he got a new notebook, fresh browser, he installed the same extension - but from a different source than the original vendor: unfortunately, this version contained malware and a big transaction was caught by compliance in the very last second, because he wasnt aware of data leakage.
You _are_ joking, right? I distinctly remember all sorts of dubious freewarez sites with slightly modified installers. 1997-2000 era. And anti-virus was a thing in MS-DOS even.
The industrialization of exploit generation is here IMO.
I still doubt they will hook up their brains though.