The way how they build that database and what models they use for text tokenization, embeddings generation and ranking at "internet" scale is the secret sauce that enabled them to raise more than $165M to date.
For sure this is where the internet search will be in a couple of years and that's why Google got really concerned when original ChatGPT was released. That said, don't assume Google is not already working on something similar. In fact, the main theme of their Google Next conference was about LLMs and RAG.
The spirit of the article is about how this can be achieved in a decentralized way without search engines, and instead with just your LLM and the embedding databases that it proposes that each website would publish.
A problem with this is that you still need to keep local copies of these databases that you get from crawling the web, and train your LLM to use it.
Before you invent a new protocol, look at Semantic Web (RDF et al), and Google Microformats, and...
edit: from article, "Doing this for a few urls is easy but doing it for billions of urls starts to get tricky and expensive (although not completely out of reach)" - indeed so, but we have now done embeddings for about half of those ~8 billion pages and are using them for mojeek.com.
We have an API with many features including uniquely authority and ranking scorings. Embeddings could be added.
https://www.mojeek.com/services/search/web-search-api/ used by Kagi, Meta and others. Self-disclosure; Mojeek team member.
^ For If You Didn't Read It
Here's their blog article for it: https://help.kagi.com/kagi/ai/quick-answer.html You have to fire up your bullshit detector when looking at the results, but I find it saves a good 3/4 clicks on average.
Llama 3 was trained on 15 trillion tokens, but I can download a version of that model that's just 4GB in size.
No matter how "big" your model is there is still scope for techniques like RAG if you want it to be able to return answers grounded in actual text, as opposed to often-correct hallucinations spun up from the giant matrices of numbers in the model weights.
