undefined | Better HN

0 pointstroupo5mo ago0 comments

For me, LLMs always, without fail get important details wrong.

- incessantly duplicating already existing functionality: utility functions, UI components etc.

- skipping required parameters like passing current user/actor to DB-related functions

- completely ignoring large and small chunks of existing UI and UI-related functionality like layouts or existing styles

- using ad-hoc DB queries or even iterating over full datasets in memory instead of setting up proper DB queries

And so on and so forth.

YYMV of course depending on language and project

0 comments

simonw5mo ago

Sounds to me like you'd benefit from providing detailed instructions to LLMs about how they should avoid duplicating functionality (which means documenting the functionality they should be aware of), what kind of parameters are always required, setting up "proper DB queries" etc.

... which is exactly the kind of thing this new skills mechanism is designed to solve.

troupoOP5mo ago

> Sounds to me like you'd benefit from providing detailed instructions to LLMs about how they should avoid duplicating functionality

That they routinely ignore.

> which means documenting the functionality they should be aware of

Which means spending inordinate amounts of time writing down about every single function and component and css and style which can otherwise be easily discovered by just searching. Or by looking at adjacent files.

> which is exactly the kind of thing this new skills mechanism is designed to solve.

I tried it yesterday. It immediately duplicated functionality, ignored existing styles and components, and created ad-hoc queries. It did feel like there were fewer times when it did that, but it's hard to quantify.

j / k navigate · click thread line to collapse