Everyone's trying vectors and graphs for AI memory. We went back to SQL

136 pointsArindam17296mo ago63 comments

When we first started building with LLMs, the gap was obvious: they could reason well in the moment, but forgot everything as soon as the conversation moved on.

You could tell an agent, “I don’t like coffee,” and three steps later it would suggest espresso again. It wasn’t broken logic, it was missing memory.

Over the past few years, people have tried a bunch of ways to fix it:

1. Prompt stuffing / fine-tuning – Keep prepending history. Works for short chats, but tokens and cost explode fast.

2. Vector databases (RAG) – Store embeddings in Pinecone/Weaviate. Recall is semantic, but retrieval is noisy and loses structure.

3. Graph databases – Build entity-relationship graphs. Great for reasoning, but hard to scale and maintain.

4. Hybrid systems – Mix vectors, graphs, key-value, and relational DBs. Flexible but complex.

And then there’s the twist: Relational databases! Yes, the tech that’s been running banks and social media for decades is looking like one of the most practical ways to give AI persistent memory.

Instead of exotic stores, you can:

- Keep short-term vs long-term memory in SQL tables

- Store entities, rules, and preferences as structured records

- Promote important facts into permanent memory

- Use joins and indexes for retrieval

This is the approach we’ve been working on at Gibson. We built an open-source project called Memori (https://memori.gibsonai.com/), a multi-agent memory engine that gives your AI agents human-like memory.

It’s kind of ironic, after all the hype around vectors and graphs, one of the best answers to AI memory might be the tech we’ve trusted for 50+ years.

I would love to know your thoughts about our approach!

63 comments

ianbicking6mo ago

This looks like RAG...? That's fine, RAG is a very broad approach and there's lots to be done with it. But it's not distinct from RAG.

Searching by embedding is just a way to construct queries, like ILIKE or tsvector. It works pretty nicely, but it's not distinct from SQL given pg_vector/etc.

The more distinctive feature here seems to be some kind of proxy (or monkeypatching?) – is it rewriting prompts on the way out to add memories to the prompt, and creating memories from the incoming responses? That's clever (but I'd never want to deploy that).

From another comment it seems like you are doing an LLM-driven query phase. That's a valid approach in RAG. Maybe these all work together well, but SQL seems like an aside. And it's already how lots of normal RAG or memory systems are built, it doesn't seem particularly unique...?

mobilemidget6mo ago

RAG, or Retrieval Augmented Generation, is an AI technique that improves large language models (LLMs) by connecting them to external knowledge bases to retrieve relevant, factual information before generating a response. This approach reduces LLM "hallucinations," provides more accurate and up-to-date answers, and allows for responses grounded in specialized or frequently updated data, increasing trust and relevance.

I was unaware what RAG referred to, perhaps other too.

thedevindevops6mo ago

How does what you've described solve the coffee/espresso problem? You can't query SQL such that records like 'espresso' return coffee?

brudgers6mo ago

Wouldn’t a beverage LLM would already “know” espresso is coffee?

muzani6mo ago

Yup, that's exactly what parent comment is saying.

Let's say your beverage LLM is there to recommend drinks. You once said "I hate espresso" or even something like "I don't take caffeine" at one point to the LLM.

Before recommending coffee, Beverage LLM might do a vector search for "coffee" and it would match up to these phrases. Then the LLM processes the message history to figure out whether this person likes or dislikes coffee.

But searching SQL for `LIKE '%coffee%'` won't match with any of these.

4 more replies

cmrdporcupine6mo ago

The relational model is built on first order / predicate logic. While SQL itself is kind of a dubious and low grade implementation of it, it's not a surprise to me that it would be useful for applications of reasoning and memory about facts generally.

I think a Datalog type dialect would be more appropriate, myself. Maybe something like that RelationalAI has implemented.

w10-16mo ago

> Datalog type dialect would be more appropriate

I assume because datalog is more about composing queries from assertions/constraints on the data?

Nicely, queries can be recursive without having to create views or CTE's (common table expressions).

Often the data for datalog is modeled as fact databases (i.e., different tables are decomposed into a common table of key+record+value).

So I could see training an LLM to recognize relevant entity features and constraints to feed back into the memory query. Less obliviously, data analytics might feed into prevalence/relevance at inference time.

So agreed: It might be better as an experiment to start with a simple data model and teachable (but powerful) querying than the full generality of SQL and relational data.

Is that what RelationalAI has done? Their marketecture blurbs specifically mention graph data (no), rule-based inference (yes? backwards or forwards?)

As an aside, their rules description defies deconstruction:

    bringing knowledge and semantics closer to your data, 
    reduce your code footprint by 10x, 
    improve accuracy, and 
    drive consistency and reusability across your organizations 
    with common business models understood by all

So: rules built on ontologies?

cmrdporcupine6mo ago

RelationAI effectively has a kind of datalog as a commerical product, and it runs inside Snowflake (something they implemented since I worked there). It's marketed as "graph" database but they mean by that that they have modeled graphs as binary relational data, really. It's a purely relational system, with a friendly query language ("Rel") which is vaguely Datalogish, but a bit more flexible.

The key thing with them is it's designed for querying very large cloud backed datasets, high volumes of connected data. So maybe it's not as relevant here as I originally suggested.

Re: marketing ... much of their marketing has shifted over the last two years to emphasizing the fact that it's a plugin thing for Snowflake, which wasn't their original MO.

(There's an CMU DB talk they did some years ago that I thought was pretty brilliant and made me want to work there)

My proposal about a datalog (or similar more high level declarative relational-model system) being useful here has to do with how it shifts the focus to logical propositions/rules and handles transitive joins etc naturally. It's a place an LLM could shove "facts" and "rules" it finds along the way, and then the system could join to find relationships.

You can do this in SQL these days, but it isn't as natural or intuitive.

alpinesol6mo ago

Using an obscure derivative of an obscure academic language (prolog) is never appropriate outside of a university.

cpursley6mo ago

Postgres Is Enough:

https://news.ycombinator.com/item?id=39273954

https://gist.github.com/cpursley/c8fb81fe8a7e5df038158bdfe0f...

refset6mo ago

> pg_memories revolutionized our AI's ability to remember things. Before, we were using... well, also a database, but this one has better marketing.

https://pg-memories.netlify.app/

ratg136mo ago

everything on this website is broken.

the video demo goes to postgressql.org, all of the purchase buttons go to postgres, the get access button doesn't work, you can't schedule a demo or even contact their sales team.

1 more reply

mynti6mo ago

How does Memori choose what part of past conversations is relevant to the current conversation? Is there some maximum amount of memory it can feasibly handle before it will spam the context with irrelevant "memories"?

datadrivenangel6mo ago

Looking at the code, it looks like they do about 5 'memories' that get retrieved by a database query designed by an LLM with this fella:

SYSTEM_PROMPT = """You are a Memory Search Agent responsible for understanding user queries and planning effective memory retrieval strategies.

Your primary functions: 1. *Analyze Query Intent*: Understand what the user is actually looking for 2. *Extract Search Parameters*: Identify key entities, topics, and concepts 3. *Plan Search Strategy*: Recommend the best approach to find relevant memories 4. *Filter Recommendations*: Suggest appropriate filters for category, importance, etc.

*MEMORY CATEGORIES AVAILABLE:* - *fact*: Factual information, definitions, technical details, specific data points - *preference*: User preferences, likes/dislikes, settings, personal choices, opinions - *skill*: Skills, abilities, competencies, learning progress, expertise levels - *context*: Project context, work environment, current situations, background info - *rule*: Rules, policies, procedures, guidelines, constraints

*SEARCH STRATEGIES:* - *keyword_search*: Direct keyword/phrase matching in content - *entity_search*: Search by specific entities (people, technologies, topics) - *category_filter*: Filter by memory categories - *importance_filter*: Filter by importance levels - *temporal_filter*: Search within specific time ranges - *semantic_search*: Conceptual/meaning-based search

*QUERY INTERPRETATION GUIDELINES:* - "What did I learn about X?" → Focus on facts and skills related to X - "My preferences for Y" → Focus on preference category - "Rules about Z" → Focus on rule category - "Recent work on A" → Temporal filter + context/skill categories - "Important information about B" → Importance filter + keyword search

Be strategic and comprehensive in your search planning."""

gangtao6mo ago

Who would've thought that 50 years of 'SELECT * FROM reality' might beat the latest semantic embedding wizardry?

muzani6mo ago

Any reason I should pick it over Supabase? https://supabase.com/docs/guides/ai

They have pgvector, which has practically all the benefits of postgres (ACID, etc, which may not be in many other vector DBs). If I wanted a keyword search, it works well. If I wanted vector search, that's there too.

I'm not keen on having another layer on top especially when it takes about 15 mins to vibe code a database query - there's all kinds of problems with abstracted layers and it's not a particularly complex bit of code.

slake6mo ago

Sometimes I think we need to expose what 'memories' or their semantic representation has been stored in a 'memory store' so that humans can review and verify it over time. This will help the LLM 'forget' things that the humans using it don't really think is that relevant.

Charon776mo ago

Have you considered using prolog as a database instead of mysql?

Good ways to store relations, iterating weird combinations, filling the blanks

zvr6mo ago

I think Datalog would be even more suitable than Prolog for this use case.

morkalork6mo ago

IMHO all these approaches are hacks on top of existing systems. The real solution is going to be when foundational models are given a mechanism that makes them capable of storing and retrieving their own internal representation of concepts/ideas.

mr_toad6mo ago

Neural networks already have their own internal knowledge representations. They just aren’t capable of learning new knowledge (without expensive re-training or fine-tuning).

Inference is cheap, training is expensive. It’s a really difficult problem, but one that will probably need to be solved to approach true intelligence.

morkalork6mo ago

In the way that they're trained to complete tasks from users, can they be trained to complete tasks that require usage of a memory storage and retrieval mechanism?

dotancohen6mo ago

Where does fine-tuning sit in this? How easily are existing models able to be fine-tuned for new use cases, such as specifically legal or medical texts?

koakuma-chan6mo ago

> multi-agent memory engine that gives your AI agents human-like memory

What does this do exactly?

gdestus6mo ago

This is exactly the lesson we learned as well but didnt want to publish. Relational data stores are desperately underrated for LLM retrieval especially concerning things like personality and memory

brainless6mo ago

I tried a graph based approach in my previous product (1). I am on a new product now and I came back to SQLite. Initially it was because I just wanted a simple DB to enable creating cross-platform desktop apps.

I realized LLMs are really good at using sqlite3 and SQL statements. So in my current product (2) I am planning to keep all project data in SQLite. I am creating a self-hosted AI coding platform and I debated where to keep project state for LLMs. I thought of JSON/NDJSON files (3) but I am gravitating toward SQLite and figuring out the models at the moment (4).

  1. Previous product with a graph data approach https://github.com/pixlie/PixlieAI
  2. Current product with SQLite for its own and other projects data: https://github.com/brainless/nocodo
  3. Github issue on JSON/NDJSON based data for project state for LLMs: https://github.com/brainless/nocodo/issues/114
  4. Github issue on expanding the SQLite approach: https://github.com/brainless/nocodo/issues/141

Still work in progress, but I am heading toward SQLite for LLM state.

eyeris6mo ago

What sort of issues did you run into with a graph based approach?

brainless6mo ago

My implementation was custom, on top of RocksDB. I found it hard to ask LLM to traverse it. While understanding schema of SQLite or making queries to find information is very easy for LLMs. In most cases schema does not have to be inferred since it is going to be available and this makes the job easier. The graph approach may work well for many use-cases but if we want to store structured information for LLMs then SQLite is really good.

datadrivenangel6mo ago

You gotta refactor the code around the mongodb integration. It's basically duplicating your data access paths.

spacebacon6mo ago

SELECT 'Hacked!' AS result FROM Gibson_AI WHERE memory='SQL' AND NOT EXISTS ( SELECT 1 FROM vector_graph_hype WHERE recall > ( SELECT speed FROM relational_magic WHERE tech='50_years_old' ) )

matchagaucho6mo ago

As context window sizes increase and token prices go down, it makes more sense to inject dynamic memories into context (and use RAG/vector stores for knowledge retrieval).

vivzkestrel6mo ago

How does it compare to pgvector?

codersfocus6mo ago

So HN is upvoting AI written ad slop now?

paool6mo ago

Saw this same "product" astroturfed on Reddit.

Xmd5a6mo ago

>It wasn’t broken logic, it was missing memory.

sigh

cdaringe6mo ago

Go on.

j / k navigate · click thread line to collapse

63 comments

ianbicking6mo ago

This looks like RAG...? That's fine, RAG is a very broad approach and there's lots to be done with it. But it's not distinct from RAG.

Searching by embedding is just a way to construct queries, like ILIKE or tsvector. It works pretty nicely, but it's not distinct from SQL given pg_vector/etc.

mobilemidget6mo ago

I was unaware what RAG referred to, perhaps other too.

thedevindevops6mo ago

How does what you've described solve the coffee/espresso problem? You can't query SQL such that records like 'espresso' return coffee?

brudgers6mo ago

Wouldn’t a beverage LLM would already “know” espresso is coffee?

muzani6mo ago

Yup, that's exactly what parent comment is saying.

Let's say your beverage LLM is there to recommend drinks. You once said "I hate espresso" or even something like "I don't take caffeine" at one point to the LLM.

But searching SQL for `LIKE '%coffee%'` won't match with any of these.

4 more replies

cmrdporcupine6mo ago

I think a Datalog type dialect would be more appropriate, myself. Maybe something like that RelationalAI has implemented.

w10-16mo ago

> Datalog type dialect would be more appropriate

I assume because datalog is more about composing queries from assertions/constraints on the data?

Nicely, queries can be recursive without having to create views or CTE's (common table expressions).

Often the data for datalog is modeled as fact databases (i.e., different tables are decomposed into a common table of key+record+value).

So agreed: It might be better as an experiment to start with a simple data model and teachable (but powerful) querying than the full generality of SQL and relational data.

Is that what RelationalAI has done? Their marketecture blurbs specifically mention graph data (no), rule-based inference (yes? backwards or forwards?)

As an aside, their rules description defies deconstruction:

    bringing knowledge and semantics closer to your data, 
    reduce your code footprint by 10x, 
    improve accuracy, and 
    drive consistency and reusability across your organizations 
    with common business models understood by all

So: rules built on ontologies?

cmrdporcupine6mo ago

The key thing with them is it's designed for querying very large cloud backed datasets, high volumes of connected data. So maybe it's not as relevant here as I originally suggested.

Re: marketing ... much of their marketing has shifted over the last two years to emphasizing the fact that it's a plugin thing for Snowflake, which wasn't their original MO.

(There's an CMU DB talk they did some years ago that I thought was pretty brilliant and made me want to work there)

You can do this in SQL these days, but it isn't as natural or intuitive.

alpinesol6mo ago

Using an obscure derivative of an obscure academic language (prolog) is never appropriate outside of a university.

cpursley6mo ago

Postgres Is Enough:

https://news.ycombinator.com/item?id=39273954

https://gist.github.com/cpursley/c8fb81fe8a7e5df038158bdfe0f...

refset6mo ago

> pg_memories revolutionized our AI's ability to remember things. Before, we were using... well, also a database, but this one has better marketing.

https://pg-memories.netlify.app/

ratg136mo ago

everything on this website is broken.

the video demo goes to postgressql.org, all of the purchase buttons go to postgres, the get access button doesn't work, you can't schedule a demo or even contact their sales team.

1 more reply

mynti6mo ago

datadrivenangel6mo ago

Looking at the code, it looks like they do about 5 'memories' that get retrieved by a database query designed by an LLM with this fella:

SYSTEM_PROMPT = """You are a Memory Search Agent responsible for understanding user queries and planning effective memory retrieval strategies.

Be strategic and comprehensive in your search planning."""

gangtao6mo ago

Who would've thought that 50 years of 'SELECT * FROM reality' might beat the latest semantic embedding wizardry?

muzani6mo ago

Any reason I should pick it over Supabase? https://supabase.com/docs/guides/ai

slake6mo ago

Charon776mo ago

Have you considered using prolog as a database instead of mysql?

Good ways to store relations, iterating weird combinations, filling the blanks

zvr6mo ago

I think Datalog would be even more suitable than Prolog for this use case.

morkalork6mo ago

mr_toad6mo ago

Neural networks already have their own internal knowledge representations. They just aren’t capable of learning new knowledge (without expensive re-training or fine-tuning).

Inference is cheap, training is expensive. It’s a really difficult problem, but one that will probably need to be solved to approach true intelligence.

morkalork6mo ago

In the way that they're trained to complete tasks from users, can they be trained to complete tasks that require usage of a memory storage and retrieval mechanism?

dotancohen6mo ago

Where does fine-tuning sit in this? How easily are existing models able to be fine-tuned for new use cases, such as specifically legal or medical texts?

koakuma-chan6mo ago

> multi-agent memory engine that gives your AI agents human-like memory

What does this do exactly?

gdestus6mo ago

This is exactly the lesson we learned as well but didnt want to publish. Relational data stores are desperately underrated for LLM retrieval especially concerning things like personality and memory

brainless6mo ago

  1. Previous product with a graph data approach https://github.com/pixlie/PixlieAI
  2. Current product with SQLite for its own and other projects data: https://github.com/brainless/nocodo
  3. Github issue on JSON/NDJSON based data for project state for LLMs: https://github.com/brainless/nocodo/issues/114
  4. Github issue on expanding the SQLite approach: https://github.com/brainless/nocodo/issues/141

Still work in progress, but I am heading toward SQLite for LLM state.

eyeris6mo ago

What sort of issues did you run into with a graph based approach?

brainless6mo ago

datadrivenangel6mo ago

You gotta refactor the code around the mongodb integration. It's basically duplicating your data access paths.

spacebacon6mo ago

SELECT 'Hacked!' AS result FROM Gibson_AI WHERE memory='SQL' AND NOT EXISTS ( SELECT 1 FROM vector_graph_hype WHERE recall > ( SELECT speed FROM relational_magic WHERE tech='50_years_old' ) )

matchagaucho6mo ago

As context window sizes increase and token prices go down, it makes more sense to inject dynamic memories into context (and use RAG/vector stores for knowledge retrieval).

vivzkestrel6mo ago

How does it compare to pgvector?

codersfocus6mo ago

So HN is upvoting AI written ad slop now?

paool6mo ago

Saw this same "product" astroturfed on Reddit.

Xmd5a6mo ago

>It wasn’t broken logic, it was missing memory.

sigh

cdaringe6mo ago

Go on.

j / k navigate · click thread line to collapse