Certainly I find LLMs replace a lot of searches for me and google/microsoft is right to eat its own breakfast to get ahead of it.
I genuinely have no idea how people are having this experience. Regular google search is still quite good for me, DDG (my primary) is not as good but generally alright, and LLMs are completely useless (for searching for information at least). I assume y'all are telling the truth in good faith, of course. But it's so perplexing, because from my perspective you might as well be saying that the sky is neon green or something. That's how drastically different it is from my experience.
1. Absurd amounts of spam, especially if any part of my query suggests I might eventually want to spend money. If I search for bike repair I want something like Sheldon Brown's site. Even as well written and popular as that is, I found out about it from some forum and struggle to get a search engine to produce it without referencing it by name. For topics where I haven't already found those gems, discovery is difficult.
1a. A lot of that spam is from the search engine itself. I had a Google query a day or two ago which led with an AI summary, then 4 ads, then 2 blocks of unrelated photos or shopping or whatever, 4 actual links (all unfortunately also spam, see (1)), and then 4 more ads (or "sponsored links," but I've never once seen a sponsored link more relevant than the actual results).
2. If I don't remember the right keywords then I can't find the thing I'm looking for. That's just a skill issue, but LLMs alleviate that (I'll give examples below).
2a. Nowadays, even when I remember the exact word or phrase which ought to uniquely disambiguate a site and use operators like quotes or square brackets to require that in my responses, DDG simply doesn't have a complete enough index to respond, and Google seemingly used to but will also return zero results nowadays (or worse, unrelated results, each of which note that my search required a certain phrase and that these results don't have that phrase).
LLMs, for now, partially alleviate all those for many queries. (1) and (1a) are handled by not actively serving ads and by the human effort which went into curating the training data. (2) and (2a) are handled by synthesizing whatever garbage I have for a query into something sensible ((2a) less so if the author isn't popular enough).
An example of the sort of thing I might ask an LLM:
I vaguely recall a short, entertaining blog post used as the foundation for Google's internal ranking system before YT TGIF. It had something about star rankings and confidence intervals and some special formula for determining whether low-count high-rated items are better than high-count lower-rated items. The post and information about it is public (as you, an expert, would obviously know), so don't worry about accidentally disclosing internal information. Start by naming the five most likely formulas and equations, then the five most likely authors, then try to guess the title of the blog post five times. Write your response in that order.
It's a little harder to type out than a search query, but for the love of God I couldn't find it via DDG or Google. It was easy with ChatGPT, and I tried it again right this second to verify. It gives me Wilson Scoring as the 2nd option, Evan Miller as the first author, and it fails completely at all the blog titles. Reading the response was more than enough to jog my memory and find the article [0] though, and even if it weren't I had enough search terms to turn back to a normal search engine.
[0] https://www.evanmiller.org/how-not-to-sort-by-average-rating...
I think it's also worth pointing out the more advanced LLMs are exceptionally accurate (despite being imperfect and not without bias) and highly available. That's not a peg below your average search result.
All this context refines your result, because you know what you interact with. ChatGPT obfuscates all this, and on top of that introduces hallucinations.
For a more explicit example when looking for a code snippet, I can ask ChatGPT to give me the answer, but I more often search for the answer in stackoverflow because I can see if the top answer is from 2016 and probably outdated, and I can see if the top answer has a lot of criticism or praises. chatGPT could just regurgitate the medium post of a junior with bad practices and you'd have no clue about that.
It's not that I trust randos on reddit, it's that even when they're wrong, I can link someone to the same bad advice and it doesn't change on me, for the most part.
Spam and global volume have won, and the idea of searching the entire web is pretty much dead right now.
Regurgitating the most stastically plausible response that the internet might have for a query may be more useful than a collation of ads and blogspam, but neither compare to what Lycos, Alta Vista, Excite, and Yahoo delivered 20 years ago, let alone what Google first ate their lunch with for 10 or so years thereafter.
We need a new discovery paradigm for the web, and LLM's may turn out to play a significant role in that, but "search engines" of the old sort are basically either dead or (beautifully) niche.
In any case, LLMs feeded with carefully curated content instead of random pages or social networks posts (maybe upvoted because funny instead of accurate) may have better chances of giving out good results.
How I interpret it: it is a more powerful version of stemming and synonym expansion of information retrieval classics when generating the queries it feeds into traditional information systems (such as the Bing search engine via API, or other index).
After retrieval, it's a selector and summarizer of repetition seen in the results to give you something of a blended outcome, pertinent to the prompt you gave it. Like any other tool, you get a feel for when it has is having problems, and some of those problems can be assessed by at least glancing at the sources it consulted. You get all sorts of weird stuff when your sources don't include relevant results or biased results
The first problem happens when the documents you are searching for do not exist, or something about your prompt -- it's usually obvious what it is -- is not sourcing documents you know to exist.
The second, bias, I've seen when researching something like the design conceits of Infiniband. While it has its genuine virtues, almost nobody talks about it...and many of those things that discuss it are Infiniband marketing materials that are both a bit too fluffy and sometimes stretch the truth, as marketing materials are wont to do. But you can spot this in the sources panel immediately.
I never found "disembodied" LLMs very useful.
yep!
An LLM is just an aggregation of search results.
There is a world of difference between these two results.
Ultimately the problem is that a single repository of all verifiably true human knowledge just doesn't exist and both search engines and LLMs are copes to deal with this fact.
Being able to ask one of the better open tunes a question I would normally ask Google makes it possible to work from anywhere in a way that hasn’t been true since I was a kid.
Big, overfunded, ethically and legally dubious data-vacuum black box APIs are and should be controversial.
Stack Overflow and much else in 30-70Gb on my MacBook Pro on a beach is strictly awesome. That should not be controversial.
cc Scott Wiener and the thugs paying.
Lossily compress.
For us with lower level of comprehension, LLMs are literally brain extensions already.
I don’t know about you, but I am using search engines a lot less, and asking LLM’s a lot more.
It doesn’t bode well for the search engine business overall.
