Ask HN: How do you send large sets of data to an LLM

2 pointsobayesshelton1y ago2 comments

So, I am hitting limits with the amount of data I am sending to Claude via the API.

I am trying to send about 5000 comments but I would love to send more but I am hitting limits of the message size.

What are some good design patterns when sending large sets of data to an LLM?

I ideally need to send all the data together at it gives context to the overall prompt.

2 comments

2 comments · 1 top-level

curious_curios1y ago· 1 in thread

Some approaches we’ve used:

- Group the comments by theme, then pass groups to the LLM to summarize/deduplicate then pass the outputs of that as context.

- RAG where only relevant parts are included in the context.

- Use an LLM with a larger context window (like Gemini pro)

Yeah, I think I need to use Gemini

Question, how if possible could you query rows in a table?

Surely the better approach would be to have some sort of connection to table rows?

j / k navigate · click thread line to collapse