I doubt either of those wishes are going to come true though. Search engines are likely always going to be SEO'ed into uselessness and GPT isn't intentionally telling lies.
Stop using GPT as a database! GPT is far more useful a reasoning engine that can accumulate fuzzy data and then provide various views or transformation of that data.
So asking GPT to parse a Wikipedia page and then asking it to teach you from it - this is a much more successful usage than what the author in the original article is doing.
It is not useful as an accurate source of information. It’s inaccurate sometimes, and it’s hard to tell when. OTOH, as a formatted, it has some actual world-changing potential.
Reasoning is not its strong point, IMO. It's a next-word prediction model, why would it be? It's doing what an LLM does. Frequently, nonsensically.
Every second that I didn't have to:
- go look up a similar problem on Google
- open three or four stack overflow tabs
- read through the stack overflow links
- copy out the answer
- change all the variables to match variable names that I want
All of this represents a huge time saver for me. I'm honestly baffled that people lack the ability to use LLMs in a productive and optimal manner.
If you want to make accurate predictions, you'll need reasoning or at least some process that approximates it.
"There a no bananas. Max is looking for bananas. Will he be successful: [yes/no]"
So are you. https://ceoln.wordpress.com/2023/03/31/its-just-predicting-t...
Tbh, it isn't surprising that it fails, its surprising that it has subjects where it is very useful.
My team has been working on several advanced techniques using reasoning on LLMs. The stacked performance of all of these techniques combined yields is quite impressive.
Hilarious to read this advice when GPT, by it's own designers, cannot reason.
Look up the Sam Altman podcast with Lex. He specifically talks about reasoning engines.
I find GPT4 useful, but I still feel its wishful thinking to call it a reasoning engine. It is, in the end, a giant model for predicting the next word in a corpus which you condition with inputs. I find it most useful to think of it as exactly that.
My suggestion is to stick with it and get a feel for what it's good at.
I've found that after a few months of using ChatGPT every day I've developed a pretty solid intuition for which questions are likely to get good answers and which are likely to trigger hallucinations.
It's difficult to describe what those intuitions are though!
One rule of thumb I've developed: if something is likely to be "common knowledge" - if it's something that is likely to have been discussed accurately on the internet by many different people - then ChatGPT is very likely to answer questions about it accurately.
If so, this information is already easy to find, making GPT redundant.
I asked chat gpt to review the topic for me and it wrote a helpful summary that tracked with what I remembered and provided enough keywords that provided better search results - fact checking and digging for more detail became much easier when I was able to find the exact wikipedia page I needed (and other resources).
You can ask it to "combine" knowledge in "novel" ways that are not discussed verbatim on the web. It's not groundbreaking reasoning by any means, but it can be very useful. ("My ridiculously specific question about model X83844-QQ combined with random factor X")
Uh, and sooo boringly, especially if there's even any tiny part of it that is developing or theoretical, and you want to learn about that part.
I've used the phrase/request "more obscure (perspectives/explanations/etc)" with GPT so many times.
(Try it, you might be surprised at alternative takes on things, takes which are not even necessarily conspiracy theories and such)
...Which has made me think: Maybe life is more boring, the more one thinks there are just really good one-and-done answers to most everything.
One of my favourite is to ask it to explain with analogies. The other day I was digging into the attention mechanism used to train LLMs, so I asked it:
"Explain queries, keys, and values in the context of LLM attention using analogies from Terry Pratchett's Discworld"
I find sometimes I get some real gems out of this that help me remember things much more effectively than just reading the basic explanation.
(In case anyone's curious I ran that just now and got the following: https://gist.github.com/simonw/777b1d19f36beb39fb4216a0238fe... )
Most everything has been covered in logical extensions of concepts started in the 80s-90s. Those products aren't good for our..
Ah there's no point commenting here anymore. HN is so blinkered in it's thinking.
The feeling of flying high on lofty concepts and pretending to get a bird's eye view of tech is no longer worth the squeeze. The thought patterns here are predictable like slashdot. I don't belong here. Bye.
This ^. Prompting google is much more intuitive than prompting a chat bot. Also results are instantly available and you get more options to chose from. You can also filter out information much easier instead of having it summarised by a closed box that decides what's best for you.
Its unclear to me how long we'll have before LLM Engine Optimization is a thing and OpenAI/MSFT "need" to turn on their LLM profitability spigot; and what ChatGPT will look like then.
That said, I'm curious as to whether technically LLMs are inherently more challenging to game than search engines.
see: windows 11
ChatGPT’s current super power is helping people get from 0-to-1 on a new topic. In particular if that topic is adjacent to or a different niche with your expertise.
It’s not currently amazing at taking someone from intermediate to advanced knowledge.
At least in my experience. If I’m using a new library/framework/API for the first time it’s amazing at answering the endless newbie questions I have.
Personally, trying to use it to write code (primarily Elixir backend and Rust systems/CLI), I tend to run into:
- Hallucinating APIs that don't exist
- Hallucinating entire libraries, despite being told repeatedly they don't exist
- Saying it will make requested changes and not doing so
- Not being anywhere close to idiomatic code
- Not being able to explain code it writes
- Running out of "memory" (I can't remember the right term. Context?) in the middle of generating code, then telling me I never prompted it when I ask it to continue
On the other hand, I've found that it's good at cleaning up ugly data. I can copy/paste in a table with bad formatting, ask it to turn it into code, and it does it near-perfectly. That's been the best use-case for it I've found so far.
I use boring normal free ChatGPT so maybe it's on me for not using GPT-4 or some other model, but either way, imo it's not been very impressive in the problem spaces I find myself in.
You can use GPT-4 fairly cheap if you sign up for a developer account on platform.openai.com and then use their playground. There you pay per usage; even though I've been using it fairly heavily, my typical monthly usage is still way under $20.
Here's an example that both impressed me and saved me a load of time just yesterday:
I've actually found it (ChatGPT-4) reasonably good at explaining other people's code, for what it's worth. It does require providing some context for the code it's explaining. ("This function is part of library X that does Y, and it seems to be about Z. I can't figure out the purpose of the loop in the middle, though. Can you explain it?")
I'm with you on it writing code, though. It makes things up too frequently to be really useful.
ChatGPT is also much better for Python and JavaScript than Rust and Elixir, presumably because it's seen an order of magnitude more example code for those languages.
Some of the “issues” are more general, as in it giving you dated answers. I asked it so build some ODATA things in Typescript and C#, and it did to varying degrees of success, but some of the code was deprecated. Some ranging to “you should never, ever, do things this way” to “well IActionResult was replaced by ActionResult but it hardly matters”.
Others was where it made things up once pressed upon being wrong. I think the two most hilarious situations was when it did its “Sorry, you’re right…” thing and then proceed to give the exact same answer it had just given prior. The other was when it made up a function that had never existed, it has a very convincing name and at first I thought it was again a matter of something deprecated, but it turned out to be completely made up code. Some of this is down to me not prompting it right, but part of it is also worrying. Because what really made me worry about it was when I joked with it. I read (listen to) a lot of Warhammer audiobooks from Black Library, and, when I jokingly asked it something silly about Khorne at one point, it led me down a rabbit hole of discussing books with it. Books it obviously had never read, but would still confidently tell you about wrongly. Maybe it got its knowledge from the internet, maybe it made it up, but what was interesting to me about it was that if it would be so confidently incorrect about those books, then what else would it be confidently incorrect about.
This isn’t just a GPT problem of course. If you want to learn something, you need to consider the sources you use. When I went to folkeskolen (school for children aged 6-14) we were taught the pyramids were build by slaves. Later that has been disputed because of how well fed the workers were, but if you wanted to learn about the pyramids and you read my old school books then you wouldn’t get the updated knowledge. Similarly a lot of the sources you can find for learning programming are outright terrible, but outside of GPT you tend to be presented with a myriad of choice to remind you that some sources are better than others. With GPT the source or what it teaches you isn’t obvious. If you wanted to learn about the pyramids, you probably wouldn’t pick a 30 year old school book for children after all.
In an ideal world, people would learn only from high quality sources like books and schools. But a shocking amount of people learn mostly through social media and whatever they find on Google.
I think LLMs will provide a better alternative.
OP learns in a way that's very child-like. When you are a five-year-old it's okay to learn by asking everything. That stops being acceptable by the age of fifteen. OP hasn't learned any research skills yet, and when OP's needs inevitably exhausts the ability of LLMs, OP would be utterly unable to read an encyclopedia or a research paper or perhaps a textbook.
I completely agree that for the latter goal, the approaches in the blog post are insufficient, even undesirable. And I do worry that the way I engage with content on the web is weakening my ability to go deep on a subject I'm interested in.
But I do think there is value in just being able to indulge curiosity quickly and consistently. Not only is it rewarding in its own right, but it also provides the spark that leads you to eventually go deeper.
Lately, I've found myself sitting at a laptop with friends, asking GPT a question, reading and discussing the response, and then coming up with and asking followup questions as a group. I don't think we would've done that in the past, because the interface of search engines and webpages and browser tabs were too unwieldy to engage with collectively. It just feels like a completely new way to learn things, and what's what I'm most excited about.
If OP reads your comment, they will be no better at learning than they were before. In that way, it's a pretty unhelpful comment.
If I want to learn about topic C which requires knowledge of topics A and B, but C can also be generalized to concepts X and Y, it will be very hard to learn from Wikipedia.
If I don't know how to add numbers and look up "sum" on Wikipedia, in the second sentence I learn that summing is used for functions, vectors, matrices, and other things I don't know about. This is a cool feature and I love it for exploring but hate it for learning things that require a few layers of concepts to get.
Textbooks do the opposite and are awesome. An electronics textbook will take you step by step through all the concepts to get to LEDs, without "forward references" to the concepts you haven't learned yet.
The "problem" with textbooks is that it will take a while to get to the destination. LEDs might be in chapter 15 and you may not want to spend a few months going through chapters 1-14. You don't know what you will need to understand chapter 15.
But you can perhaps work backward - you are guaranteed that any unfamiliar concept introduced in chapter 15 will be covered in chapters 1-14, and that there is no rabbit hole.
ChatGPT or a personal tutor can shortcut this by giving you just the "narrow path" of knowledge to understand the concept that you want to learn.
> just out of curiosity, I wanted to learn more. I get that LEDs consume less energy and release less heat, and that they're made using semiconductors. But what kinds of semiconductors? How do semiconductors work in general, anyway?
And they proceed to type "LED" into Google. Why not "led what kind of semiconductor" and "how do semiconductors work in leds"?
I assume, OP didn't write "LED" in the ChatGPT text box without any context either.
I did try Googling “how do LEDs work” for comparison, but it yielded the same top few results. Of course, I could have iteratively tried different search queries to get to the answers I wanted, but this gets at my real point: I don’t have to formulate 5 different search queries anymore, allowing me to maintain one focused line of inquiry. I talk about this a little in the “fewer browser tabs” bit of the post.
I do think someone could create an alternative search UI that would be better for learning on the web. Something where you can run multiple searches and “collect” the useful information you find into a single page, rather than having the results split across a mess of browser tabs and note-taking windows. Maybe I just find juggling many browser tabs more annoying than other people do?
Anyway, I tried the queries you posted above, and most resources I found were still very confusing for a layman. The one exception is this page, which I think does a great job of introducing additional complexity on this topic gradually: https://electronics.howstuffworks.com/led.htm
A lot of the paper is talking to peer and people wanting to verify the validity of the paper - by it being peer reviewed, you can mostly assume that the paper is valid, and stick to what the paper is saying instead of it's methodology.
you should check this
https://en.wikipedia.org/wiki/Replication_crisis
https://en.wikipedia.org/wiki/Data_dredging or P-hacking
and many more...
What? Is that something you can still assume in 2023?
Right now, if the level involves advanced math, it’s better to switch to other sources at some point, but that will change.
You can ask GPT-4 to tutor you, also.
You can't possibly know that.
Even if the answers accidentally happen to be correct, that's just the broken clock happens to be correct twice a day. The information value of answers by ChatGPT is zero.
As I said in https://simonwillison.net/2023/Apr/8/llms-break-the-internet...
> This is the thing I worry that people are sleeping on. People who think “these language models lie to you all the time” (which they do) and “they will produce buggy code with security holes”—every single complaint about these things is true, and yet, despite all of that, the productivity benefits you get if you lean into them and say OK, how do I work with something that’s completely unreliable, that invents things, that comes up with APIs that don’t exist… how do I use that to enhance my workflow anyway?
You can't rely in it being 100% correct, but that's very different from it having no informational value at all. When it comes down to it, you can't rely on anything being 100% correct. I recall finding multiple errors in textbooks in the past, and certainly Wikipedia is wrong about all kinds of things; that doesn't make them useless. It just means in situations where it's critical to be correct, you need to double-check. But often that's not necessary, and when it is, it's a lot easier to start with something and then verify it than to not have the tool in the first place.
Nonsense. The only way this would be true is if ChatGPT's answers were as likely to be correct as random answers. They are much, much more likely to be correct, however, so their information value is greater than zero by definition.
What you're claiming is equivalent to "search engines can find incorrect information, so search engines are worthless for information retrieval". Which is bollocks.
I have not found these explanations sufficient, while I have done a bit of chemistry and physics I understand the basics of light emissions, but reading this has the same value as reading the wiki article on LEDs to me.
To learn anything useful on the internet, you pretty much have to skim. So much of the internet is so loaded with filler and BS that it is hardly worth reading at all.
With ChatGPT, it’s incredibly refreshing to be able to ask a question and get nothing other than a concise answer. No skimming required. I feel so much more focused and better able to learn this way.
Otherwise ChatGPT is prone to injecting lots of filler text as well.
GPT is like that "know-it-all" friend we have who just has something to say about anything, with knowledge skimmed from the internet.
GPT is a language model. It outputs what you want to hear, not what is correct.
Nonsense. How could it possibly know "what you want to hear"?
Maybe a better rephrasing would be "ChatGPT has been trained to give answers to questions in a clear, confident manner, regardless of the content"
From the announcement post: https://openai.com/blog/chatgpt
> We trained this model using Reinforcement Learning from Human Feedback (RLHF), using the same methods as InstructGPT, but with slight differences in the data collection setup. We trained an initial model using supervised fine-tuning: human AI trainers provided conversations in which they played both sides—the user and an AI assistant.
So I agree with OP; it's been trained to give answers that sound plausible but not necessarily correct. It's even mentioned in the "Limitations" section at the bottom of the blog post.
> ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth; (2) training the model to be more cautious causes it to decline questions that it can answer correctly; and (3) supervised training misleads the model because the ideal answer depends on what the model knows, rather than what the human demonstrator knows.
"How do I do this"
"How do I do that"
"What is this"
"What is that"
From [Make Something Wonderful: Steve Jobs in His Own Words][1], Steve once said in a interview in 1983:
> The problem was, you can't ask Aristotle a question. And I think, as we look towards the next fifty to one hundred years, if we really can come up with these machines that can capture an underlying spirit, or an underlying set of principles, or an underlying way of looking at the world, then, when the next Aristotle comes around, maybe if he carries around one of these machines with him his whole life–his or her whole life–and types in all this stuff, then maybe someday, after this person's dead and gone, we can ask this machine, “Hey, what would Aristotle have said? What about this?” And maybe we won't get the right answer, but maybe we will. And that's really exciting to me. And that's one of the reasons I'm doing what I'm doing.
And this future, expected "next fifty to one hundred years", is somewhat here already.
Because if you don't, you won't understand the answers.
That’s a positive thing about this generative AI revolution that I haven’t really thought about in those terms until now.
Agreed. Look at all the responses in this post attacking the author for how he learns, it's embarrassing to read. Now imagine that person is actually a teacher or TA, or worse a co-worker. I'd much rather deal with an imperfect ChatGPT session than that kind of flippancy.
People go on and on and on about "accuracy", completely ignoring that accuracy is irrelevant to 99.9% of things that humans do in their everyday lives.
Simulating (positive) human interaction is far more impactful than getting facts correct.
On that note, search in that regard always reminded me of those times where you ask a teacher how to spell a word and they say to look it up in the dictionary.
I think this is partly why I'm still looking to be wowed by this technology, personally, in terms of what it can accomplish for me. And while it could be rightly said I've made things unnecessarily hard for myself approaching life like this, I feel it has been beneficial, and enriching, to force myself to really ask, what is this person saying here? In particular, I wouldn't want GPT to lead to a general lessening of empathy.
I personally don’t believe there’s mutual exclusivity here.
I spent the first half of my life just absorbing. I’ve spent the 2nd half of my life so far undoing the patterns of thought that can result from staying inside one’s head instead of engaging with people.
In my experience, asking the person saying something what they mean is far more effective than asking myself what they mean.
When I ask them, I can form real empathy.
When I only ask myself, I just waste significant energy on all of the ways I imagine I could or should be empathetic.
> In particular, I wouldn't want GPT to lead to a general lessening of empathy.
While I have a lot of concerns about LLMs and the future of literacy, propagation if misinformation, etc, I don’t think ChatGPT is any more risk to empathy than 100 other aspects of modern life.
Facebook, Twitter and Reddit seem far more responsible for the erosion of empathy, and it’s unclear how LLMs would inherently lead to a “general lessening”. I think that ship sailed a decade ago.
Put another way, if you’re going to sources other than the individual speaking to clarify what they’re saying, the underlying issue is probably not ChatGPT or whatever the next tool is that comes around.
Another form of what you describe is leaning on one’s friends/acquaintances. Plenty of people do this, often with poor results. Reddit’s various relationship forums are a great example. I translate what I thought I heard and ask a 3rd party who wasn’t there what they hear. But by doing so, I remove even more context and make it even less likely to arrive at a useful answer.
I’m sure people will use LLMs for this, but the root issue is deeper, not caused by these tools.
I think that with time, we’ll get better at determining which types of conversations are worthwhile and which aren’t.
If I’m trying to understand a complex multi-faceted technical issue, it’s amazing to be able to drill deeper and deeper into the knowledge contained within the LLM.
If I’m trying to understand the internal states of other people, I have no reason to believe I’ll find good answers in a model that wasn’t trained on that person’s thoughts.
I think we need to think about how to keep these valuable sites going, because they are ultimately providing most of the value of the various available language models.
setting up a venv.. environmental variable issues in windows. *diagnosing a UTF-8 issue in windows.
i get that professionals problems would be harder to answer...however, getting responses without wading through stack exchange entry after entry has really kept me focused, and prevented the often times frustrating recursive spiral which is getting an issue with your issues issue...
It didn't give a wrong answer when asked "Is there a digital to analog converter with an 8V analog range and serial input?", which another poster (mhb) had shown to trip up plain GPT4.
Having a conversation allows me to figure out what I’m trying to figure out.
Once I know that, I know what to look for.
To me, the value of the conversation isn’t its perfect accuracy, but the expressiveness and ease of veering in any direction that seems interesting at the moment. The efficiency gains when jumping around within a subject are incredible.
For me it’s consistently been much more helpful than google.
* Various new english idioms. I read a lot, but still find new ones
* More optimal translations between languages i'm ~native in.
* ton of new JS, Python libs
* Got deeper in SQL (recursive queries, etc)
* Can use CSS way better than before
* Mermaid
I wrote https://chatcraft.org/ as a friendlier UI for learning programming related things.
I always finish up by asking GPT to test my knowledge with a single-choice questionnaire. What I've observed is that the retention of the material is higher compared to "traditional" techniques. Perhaps the conversation style is more immersive, or perhaps focusing on specific knowledge gaps makes for accelerated / personalised learning.
There is of course the problem of accuracy, but I feel like it's often over-stated. Even if GPT is not correct at times, it often uncovers concepts and relations that paint a better overall picture for me, and lead me to better questions and follow up actions.
There seems to be less next-level analysis: which topics are more prone to inaccuracy, does the critique loop actually help LLMs overcome those inaccuracies, and do the benefits of LLMs outweigh the consequences of these inaccuracies?
PS you would also need to do this if you started with Wikipedia as well.
Search engines are much better On uncovering guides on that, specially from experts and verified sources. It's a bit of work to verify that, but then it is part of learning itself, not sure trying to punt on that to fast track your learning your whatever is going to make a meaningful difference in terms of time
- summarising entire chapters (summarise chapter 8 of book 1)
- asking to verify my own summaries (in summary, smith writes that ... am i correct?)
- explaining certain passages in ever more detail (explain ...; explain in simpler terms; im still confused, explain in more details;)
---
One very interesting case is when GPT produced an error, and I had it go through the error with me. In the end GPT managed to spot its own error and correct itself. Here's an excerpt, edited for brevity:
* gpt: [...] When the demand is too low, it encourages growth; when the demand is too high, it restrains growth. [...]
* me: "When the demand is too low, it encourages growth; when the demand is too high, it restrains growth." is this correct?
* g: Apologies for the confusion in my previous response. The correct statement should be:
When the demand for labor is too low (slow), it encourages higher wages and population growth; when the demand for labor is too high (fast), it restrains population growth by adjusting wages to find the equilibrium between demand and supply.
(GPT still giving the wrong answer.)
* m: i am still confused. can you explain in more detail
* g: [...] Smith argues that the demand for labor regulates population growth by influencing wages. If wages are too low, fewer children will be born, and the population growth will be slow. If wages are too high, more children will be born, and the population growth will be fast. [...]
(At this point GPT was able to give me the correct response.)
* m: so when the demand for labour is low, wages decrease and population growth slows down. when the demand is high, wages increase and population growth speeds up. correct?
* g: Yes, that's correct. [...]
(I pointed out that this was in contradiction with the earlier response.)
* m: did you not contradict yourself? "When the demand is too low, it encourages growth; when the demand is too high, it restrains growth."
* g: Apologies for the confusion in my previous response. I misunderstood your question. Let me clarify:
When the demand for labor is too low, it restrains growth; when the demand is too high, it encourages growth.
So, when the demand for labor is low, wages decrease, and population growth slows down. Conversely, when the demand for labor is high, wages increase, and population growth speeds up. I hope this clears up any confusion.
(I then inserted the original erroneous response as a prompt, and gpt was able to "remember" that it generated the response earlier.)
* m: "When the demand is too low, it encourages growth; when the demand is too high, it restrains growth." what is wrong with this passage?
* g: I apologize for the confusion earlier. There is an error in the passage you provided. It should be:
This demand for labor operates like the demand for any other commodity, regulating the production and growth of the population. When the demand is too low, it restrains growth; when the demand is too high, it encourages growth. This dynamic determines the rate of population growth in different countries, such as rapid growth in North America, slow and gradual growth in Europe, and a stationary population in China.
---
i should mention that i am using the paid version (GPT-4). i tried the free model, but the quality was significantly worse. you can recorrect it multiple times consecutively and every time it would say that it was wrong the last time, and recorrect itself again and again.
If I want to try Rust, I don't want to be taught uint8 v uint16 or that you shadow variables. I want to know the interesting parts.
ChatGPT is pretty good at this and the other thing I want: pandas training. You can ask it to generate exercises at any difficulty and also provide test data!
This tool is the biggest mind expander for me since search engines.
I tend to get better answers from their "automated questions" which are paraphrased versions of my query. So it clearly understands what I'm after.
In order to promote diversity, i would recommend perplexity.ai which offers a similar experience as chatgpt (I'm not affiliated and i have no clue what their tech stack is like) It also offers links back to pages and follow up questions etc. Highly recommended if you need to learn something new and you don't want to bang your head on the keyboard googling or ddg'ing
I'll give an example. I recently needed to learn about k8s, minikube, kubectl et al for a project. I had some vague idea about the tech but nowhere near enough for what i needed to do. Google was useless because it kept taking me to doc pages which is like being lectured but i needed specific information. Perplexity was amazing in helping me with the right bit of information, example code AND links if i do want to read further
If I were learning how LEDs work, I would not have wasted any time whatsoever on the search results that the author spent a lot of time on. They were obviously (to me) the wrong articles on the face of it, because they were covering aspects that weren't really what I was looking for (the wrong sort of detail and emphasis).
So I think I would have been off and running pretty much immediately with the web search results rather than spending time on the clear dead ends.
ChatGPT gets me there too, after enough back-and-forth, but it takes longer for me to zero in on what I'm looking for.
I say this not to say that ChatGPT is in any way bad for this. I'm just noticing a difference in how the two of us engage in learning new topics. Perhaps the reality is that for some people, ChatGPT is a godsend, and for others, it's fine... but hardly an improvement for this use use case.
It would explain a lot.
(Also, when did learning stop being fun??)
ChatGPT addresses a scalability problem: not everyone has access to a tutor or can just call up a teacher or mentor to learn and ask questions. But some in the tech industry claim that ChatGPT is as good as or even better than human instruction, which to me seems totally off base.
The biggest problem I see in using LLMs as a teacher-substitute is that LLMs answer the questions you ask, whereas a good teacher tells you what you need to hear. Maybe this is solvable with specialized model tuning, but we need to actually solve it before telling kids that the best way to learn is to talk to the computer.
It's also magical when you summarise the understanding you've reached back to it, and it can confirm or tweak it for you.
In other ways it's also nice to just pay for it and then to be in an advertising free space.
One critique is that when you ask it to compare things it's often too balanced or too positive/enthusiastic ("both are great for different reasons!") when what you want is a more sober analysis. But you can usually do some prompt management to adjust it back to a reasonable range.
References:
Scary part is I don't know enough about the subject to tell apart truth from falshood because they are stated in exactly the same confident manner. Also most things are true with falsehoods sprinkled in between
On the other hand, I feel like my experience with Phind has been so great because with all dev stuff I ask it it's always been something I could try out right away.
With non-dev stuff that isn't always the case. I'd love an LLM that can do the same thing as Phind, as in providing the sources it's using, while also working for everything.
I'm not taking anything away from Chat-GPT for this use case, but I see the above pattern in almost every Google bashing thread on HN.
But I was bitten by a series of completely made up facts, and my skepticism has forced me to research nearly every complex response since then to confirm.
This fatigue has really shut down my enthusiasm. Maybe I'll research a topic 3-4 hours a week now.
