LLMs don't remember context well—they can’t ingest your data and keep it in memory. This limitation leads to lacking interactions, a lack of accuracy, and the inability to connect your data sources cheaply because developers must include long, unmanaged context in every prompt.
When we were building RAGs we saw that we can’t find the data we need and that there are too many knobs to turn in RAG frameworks. We had to tweak many parameters and also could not specify the rules we wanted the data to follow. No ontologies, no rulesets, no state or good data engineering practices and a lot of manual work. That’s why cognee
cognee builds memory that combines graph, vector, and relational stores. Here is how it works:
Adding data: When you use cognee with your AI App, it can take in any message, string, S3 bucket or even a relational database and automatically ingest it
Managing information: cognee sorts this information into semantic graph: - It extracts entities and connections between things. - It embeds the data in the vector store, It enriches data with custom ontologies you provide, that help ground the graphs and make them more reliable. - The overall information is stored in many layers of a graph and vector store that allows for finding similar information later using a variety of types of searches.
Retrieving data: When given an input query, cognee searches for and retrieves related stored information by leveraging a combination of graph traversal techniques, vector similarity, and COT techniques. It can use the internal benchmarking system to make sure that your pipelines are returning only accurate data when you need it.
---
cognee introduces a self-improving group of memory layers that cover various topics, data sources and can be customized. This reduces the need to build everything from scratch, and you can use our primitives to get started and move more quickly. On the other hand, you can do everything yourself, from start to end. We’ve designed the system to be modular and extensible.
We’ve open-sourced cognee —specifically the framework and various vector and graph database adapters, as well as our default data pipeline, cognify—under the Apache 2.0 license. This includes the ability to add, cognify, and retrieve data within your AI applications and also extend it with custom components that are just pure Python.
However, many keep features that are optimized for production use, as a part of our paid platform. We release these functionalities too, including in the last few weeks permission management + distributed pipelines(dev). These are a part of our open-source package and are available to those who in production environments need things like rate limiting and credential management. We will release a paid offering for developers to get easy API access to our platform.
Our automation tooling allows us to optimize the pipelines and find the best combination of parameters that answer questions our stakeholders have.
We’d love to hear what you think! Please feel free to try our demo, check out the code, read the research paper, and share thoughts or suggestions with us. Your feedback will help shape where we take cognee from here!
