- Generative AI uses the context you provide to help generate additional tokens
- If the context you provide is bad (low-quality code, riddled with security issues), you'll similarly get low-quality code generated
- If the context you provide is good (high-quality code), you'll get better-quality code out
The thing we wanted to highlight with this research is that security is meaningfully impacted when you're generating code, particularly with low-quality codebases.
If you didn't write it, then it's going to be low quality, insecure code by default.
Over time, it would start to suggest larger and larger blocks of code based on what I was writing. It got to the point where it would auto-suggest entire functions based on my code comments. e.g. I'd type: "//Update a user" and it would then suggest complete code for doing just that, while managing to match the style of previous methods, use the correct ORM objects, etc. In general, it seemed to be 90-95% "correct".
However, it did like to remind me that it was just really good at putting words together. For the Update user, it suggested a function that exactly matched how I'd write it, except for this:
//Persist changes to Show object to the database
user.Save()
Just a nice reminder that the GenAI is not "thinking" about what it's doing.
Personally, the most risky AI stuff I do is if I am completely stuck on something, I might accept AI suggestions without much thought just to see if it can resolve whatever issue I am running into. But in my mind, those parts of the code are always "dirty" until I thoroughly review them; in the vast majority of cases, I end up refactoring those parts myself. If I am asking AI to improve a text I wrote, I rarely just take it as-is, I typically open both versions next to each other and apply parts I like to my original text.
In my opinion, stuff created by AI is inherently "unfinished". I cringe whenever people have AI do something and just roll with it (writing an essay, code, graphic design, etc.). AI is excellent for going most of the way, but in most cases, there need to be review and finishing touches by a human, at least for now.
