Ask HN: Local small LLM with retries vs. STOA Frontier, will it work?

1 pointsusermac6d ago1 comments

I would just like to know, most machines don't have a ton of memory. If I put a super small LLM, and if I were to have unit test, do a loop until it passes this unit test. Is that a real possibility or is it just out of the question?

1 comments

kvisner6d ago

Depends on how hard the question is.

Simple functions in small code bases, will probably work.

Once you get large code bases and more complex work, you'd have issues with the small LLM having the context it needs to actually solve the issue.

j / k navigate · click thread line to collapse

Ask HN: Local small LLM with retries vs. STOA Frontier, will it work?

1 pointsusermac6d ago1 comments

1 comments

kvisner6d ago

Depends on how hard the question is.

Simple functions in small code bases, will probably work.

Once you get large code bases and more complex work, you'd have issues with the small LLM having the context it needs to actually solve the issue.

j / k navigate · click thread line to collapse