The maximalists and skeptics both are confusing the debate by setting up this straw man that people will be delegating to LLMs blindly.
The idea that someone clueless about OAuth should develop an OAuth lib with LLM support without learning a lot about the topic is... Just wrong. Don't do that.
But if you're willing to learn, this is rocket fuel.
It was extremely frustrating.
Before AI generated results, the first page of Google was SEO-optimised crap blogs. The internet has been hard to search for a while.
feels like there's a logical flaw here, when the issue is that LLMs are presenting the wrong information or missing it all together. The person trying to learn from it will experience Donald Rumsfield's "unknown unknowns".
I would not be surprised if we experience an even more dramatic "Cobol Moment" a generation from now, but unlike that one thankfully I won't be around to experience it.
You are getting a stylised view of a topic from an entity who lacks the deep understanding needed to be able to fully distill the information. But it is enough to gain enough knowledge for you to feel confident which is still valuable but also dangerous.
And I assure you that many, many people are delegating to LLMs blindly e.g. it's a huge problem in the UK legal system right now because of all the invented case law references.
Isn't this how every child learns?
Unless his father happens to be king of Macedonia, of course.
They are fine with a lossy, imperfect version of how the world works.
(One thing that might be useful is use the LLM as a search engine to find the relevant RFCs since sometimes it’s hard to find all of the applicable ones if you don’t know the names of them already.)
I really can’t stress this enough: read the RFCs from end to end. Then read through the code of some reference implementations. Draw a sequence diagram. Don’t have the LLM generate one for you, the point is to internalize the design you’re trying to implement against.
By this time you should start spotting bugs or discrepancies between the specs and implementations in the wild. That’s a good sign. It means you’re learning
making mistakes is how we learn, and if they are never pointed out...
Sure, having access to legit experts who can tutor you privately on a range of topics would be better, but that's not realistic.
What I find is that if I need to explore some new domain within a field I'm broadly familiar with, just thinking through what the LLM is saying is sufficient for verification, since I can look for internal consistency and check against things I know already.
When exploring a new topic, often times my questions are superficial enough for me to be confident that the answers are very common in the training data.
When exploring a new topic that's also somewhat niche or goes into a lot of detail, I use the LLM first to get a broad overview and then drill down by asking for specific sources and using the LLM as an assistant to consume authoritative material.
You know that it's possible to ask models for dissenting opinions, right? Nothing's stopping you.
> and if they are never pointed out...
They do point out mistakes though?
LLMs will tell you 1 or 2 lies for each 20 facts. Its a hard way to learn. They cant even get their urls right...
That was my experience when growing up with school also, except you got punished one way or another for speaking up/trying to correct the teacher. If I speak up with the LLM they either explain why what they said is true, or corrects themselves, 0 emotions involved.
> They cant even get their urls right...
Famously never happens with humans.
If you are in class, and you incorrectly argue, there is a mistake in an explanation of Derivatives or Physics, but you are the one in error, your Teacher hopefully, will not say: "Oh, I am sorry you are absolutely correct. Thank you for your advice.."
With learning, aren’t you exposed to the same risks? Such that if there was a typical blind spot for the LLM, it would show up in the learning assistance and in the development assistance, thus canceling out (i.e unknown unknowns)?
Or am I thinking about it wrongly?
One big technique it sounds like the authors of the OAuth library missed is that LLMs are very good at generating tests. A good development process for today’s coding agents is to 1) prompt with or create a PRD, 2) break this down into relatively simple tasks, 3) build a plan for how to tackle each task, with listed out conditions that should be tested, 3) write the tests, so that things are broken, TDD style and finally 4) write the implementation. The LLM can do all of this, but you can’t one-shot it these days, you have to be a human in the loop at every step, correcting when things go off track. It’s faster, but it’s not a 10x speed up like you might imagine if you think the LLM is just asynchronously taking a PRD some PM wrote and building it all. We still have jobs for a reason.
How do you determine if the LLM accurately reflects what the high-quality source contains, if you haven't read the source? When learning from humans, we put trust on them to teach us based on a web-of-trust. How do you determine the level of trust with an LLM?
But TDD is not the way I think. I've never been able to work that way (LLM-assisted or otherwise). I find it very hard to write tests for software that isn't implemented yet, because I always find that a lot of the details about how it should work are discovered as part of the implementation process. This both means that any API I come up with before implementing is likely to change, and also it's not clear exactly what details need to be tested until I've fully explored how the thing works.
This is just me, other people may approach things totally differently and I can certainly understand how TDD works well for some people.
This does mean that there's a reliance on me being able to determine what are key facts and when I should be asking for a source though. I have not experienced any significant drawbacks when compared to a classic research workflow though, so in my view it's a net speed boost.
However, this does mean that a huge variety of things remain out of reach for me to accomplish, even with LLM "assistance". So there's a decent chance even the speed boost is only perceptual. If nothing else, it does take a significant amount of drudgery out of it all though.
I don't think that's how things work. In learning tasks, LLMs are sparring partners. You present them with scenarios, and they output a response. Sometimes they hallucinate completely, but they can also update their context to reflect new information. Their output matches what you input.
It's important to remember that these models depend not only on ML breakthroughs but also on the breadth and freshness of the data used to train them.
That said, the "next-door" model could very well incorporate lessons from the recent Cloudflare OAuth Library issues, thanks to the ongoing discussions and community problem-solving efforts.
Just wrong. Don’t do that
I’ve grown the most when I start with things I sort of know and I work to expand my understanding.
You don't get knowledge by ONLY talking to LLMs, but they're a great tool.
Sorry, but the the amount of bad information dispensed by models and the student's ability to go "hey, that's wrong" due to a lack of experience and knowledge means that this is going to lead to disaster very often.
People already dispensing terrible information on YouTube because they trusted an AI to generate their voice-over script to explain something when creating learning materials.
We’ve gone from skeptics saying LLMs can’t code, to they can’t code well, to they can’t produce human-level code, to they are riddled with hallucinations, to now “but they can’t one-shot code a library without any bugs or flaws” and “but they can only one-shot code, they can’t edit well” even tho recents coding utilities have been proving that wrong as well. And still they say they are useless.
Some people just don’t hear themselves or see how AI is constantly moving their bar.