A cybersecurity research company can now spend a small fortune on finding zero days in iOS because of the amount of people that use it. It basically guarantees there will be clients like government agencies willing to pay through the nose for the exploits.
Software made for one might disrupt this business model.
With software that's deployed to millions of computers you have an abundance of targets, but trying to target some random LLM average todo list at scale is hard, isn't it?
Once millions of completely unskilled developers have "workflows" that consist of asking an LLM to make a thing, followed by those LLMs pulling in the same 100 (often outdated versions of) dependencies, you have a beautiful attack vector.
Yes, it's "easy" to attack something like Obsidian. It's probably easier to attack a couple hundred dependencies LLMs like to use, or to test what LLMs commonly do to implement things from scratch, and attack those weaknesses.
We are just lucky that enough real, smart, people engineered things that actually work, are well understood, and keep us safe, like firewalls.
Maybe, in fact, some group somewhere combines supply chain attacks with models and/or agents. A model or agent that’s compromised upstream and becomes designed to insert a backdoor is not beyond possibility.
At day 9, right as he is getting ready to deploy his beautifully crafted shell code, the clock hits midnight UTC. The website shuts down for maintenance.
"This is it" he thinks. "As soon as the backups finish I'm getting in. No problem."
Minutes tick by. He gets up, stretches, sits back down, watches the clock impatiently. Then, as he prepares to start refreshing the site he recollects, "I'm glad I begged so hard to get authorization to use this PHP 0day."
His partially obscured terminal window has the script ready to launch, all arguments pre-populated, waiting for the link and session token to be pasted in and executed.
The site comes back up. But the url of his launch point returns 404. Undaunted, he returns to a previous url. It is also 404. He curses aloud. Beginning to perspire, he goes to the homepage and prepares to navigate back to the launch point.
The link isn't there. Well, it's there, but it has changed.
"What the....!" The link is no longer a PHP url. He mouses over other links. NO links say PHP anymore. Starting to panic, he clicks on links at random. Not a single one appears to be PHP.
The following morning he schedules an urgent meeting with his supervisor.
"How's that project coming along. Got anything yet?"
"No. I, uh...I'm going to need a bit more time."
"Oh?"
"Yeah. Uh. The site. It got..." He mutes his microphone and, for the 22nd time since midnight, he screams in frustration. Unmuting, he continues:
"It got rewritten. Completely. In Nim."
"What??"
"Yeah. It's some esoteric language that just got a new web framework. I guess somebody decided they wanted to mess around with it. So they vibe coded a complete translation. The whole front end is nimlang now. None of the PHP attacks are going to work on it."
His supervisor expresses his disgust and ends the call.
11 days later the process repeats itself, this time with Rust.
The TAO engineer submits an application to change jobs to the DoD's procurement division, then requests an appointment with a mental health counselor.
