zhyder

Engineers are increasingly setting up coding agents to run continuously, some running multiple agents in parallel. I've struggled to do that because I've struggled to build confidence without full code review.

How are y'all reviewing all this massive code output? How can it possibly scale, as the agents run faster or as you add agents?

I guess we'll have to learn to give up some control; we'll stop reviewing all lines of code, and increasingly rely on AI tools to summarize and flag specific lines. Are any tools good at this?

More generally, how do you build confidence in the code, both at the PR level and eventually at the codebase level (when 90+% of code in it will be written by agents)?

Am I too worried about code review, are there bigger bottlenecks in our jobs when using these coding agents?

Is there any project out there that's trying to figure out how to filter the web to its (increasingly shrinking) fraction that can be trusted? Trusted as in NOT AI slop, content spam, or intentionally poorly-researched/misleading.

There are different thresholds to "trusted": the highest that'd be useful to search would be just authoritative go-to sources for something, e.g. the official docs for an API or library. But that'd be perhaps too narrow: including stackoverflow links around that API or library usage would be fine. In other domains like product reviews, you'd include only sites that have a reputation for actually testing the products.

Use-cases: - Reliably good search results... Google et al are full of SEO spam now and soon will be full of AI slop - Better grounding for LLMs than current search grounding... even if the LLM doesn't hallucinate it can cite junk on the web - Better pre-training data... I actually don't understand how LLMs will themselves filter out their own slop from future pre-training runs

I'm not sure what form this should take if it doesn't exist yet. Maybe a github project or wiki curating links per domain (yahoo directory reinvented?), each of us curating our own bookmarks and sharing it (delicious reinvented?), something else?