I kept telling them that it works well if you have a standard usage case but the second you need to something a little original you have to go through 5 layers of abstraction just to change a minute detail. Furthermore, you won't really understand every step in the process, so if any issue arises or you need to be improve the process you will start back at square 1.
This is honestly such a boost of confidence.
Most LLM applications require nothing more than string handling, API calls, loops, and maybe a vector DB if you're doing RAG. You don't need several layers of abstraction and a bucketload of dependencies to manage basic string interpolation, HTTP requests, and for/while loops, especially in Python.
On the prompting side of things, aside from some basic tricks that are trivial to implement (CoT, in-context learning, whatever) prompting is very case-by-case and iterative, and being effective at it primarily relies on understanding how these models work, not cargo-culting the same prompts everyone else is using. LLM applications are not conceptually difficult applications to implement, but they are finicky and tough to corral, and something like LangChain only gets in the way IMO.
I built an agent-based AI coding tool in Go (https://github.com/plandex-ai/plandex) and I've been very happy with that choice. While there's much less of an ecosystem of LLM-related libraries and frameworks, Go's concurrency primitives make it straightforward to implement whatever I need, and I never have to worry about leaky or awkward abstractions.
The OpenAI api and others are quite raw, and it’s hard as a developer to resist building abstractions on top of it.
Some people are comparing libraries like Langchain to ORMs in this conversation, but I think maybe the better comparison would be web frameworks. Like, yeah the web/HTML/JSON are “just text” too, but you probably don’t want to reinvent a bunch of string and header parsing libraries every time you spin up a new project.
Coming from the JS ecosystem, I imagine a lot of people would like a lighter weight library like Express that handles the boring parts but doesn’t get in the way.
I ran into similar limitations for relatively simple tasks. For example I wanted access to the token usage metadata in the response. This seems like such an obvious use case. This wasn’t possible at the time, or it wasn’t well documented anyway.
- Read in the user's input
- Use that to retrieve data that could be useful to an LLM (typically by doing a pretty basic vector search)
- Stuff that data into the prompt (literally insert it at the beginning of the prompt)
- Add a few lines to the prompt that state "hey, there's some data above. Use it if you can."
[disclaimer I created Hamilton & Burr - both whitebox frameworks] See https://www.reddit.com/r/LocalLLaMA/comments/1d4p1t6/comment... for comment about Burr.
Was driven to do so because it was not as easy as I'd like to override a prompt. You can see how they construct various prompts for the agents, it's pretty basic text/template kind of stuff
https://developers.cloudflare.com/workers-ai/tutorials/build...
I was fortunate in that the person I was building the project for was able to introduce me to a few other people more experienced with the entire nascent LLM agent field and both of them strongly steered me away from LangChain.
Avoiding going down that minefield ridden path really helped me out early on, and instead I focused more on learning how to build agents "from scratch" more or less. That gave me a much better handle on how to interact with agents and has led me more into learning how to run the various models independently of the API providers and get more productive results.
On the other hand it took some years into the web, for some web frameworks to emerge and make sense, like Ruby on Rails. Maybe in 3-4 years time, complicated chains of commands to different A.I. engines will be so difficult to get right that a framework might make sense, and establish a set of conventions.
Agents, another central feature of LangChain, are not proved to be very useful as well, for the moment.
Kudos to the LangChain folks for building what they built. They deserve some recognition for that. But, yes, I don’t think it’s been particularly helpful for quite some time.
I ended up calling the model myself and extracting things using a flexible json parser, I ended up doing what I needed with about 80 lines of code.
Langchain, Pinecone, it’s all the same playbook.
I appreciate Fabian and the Octomind team sharing their experience in a level-headed and precise way. I don't think this is trying to be click-baity at all which I appreciate. I want to share a bit about how we are thinking about things because I think it aligns with some of the points here (although this may be worth a longer post)
> But frameworks are typically designed for enforcing structure based on well-established patterns of usage - something LLM-powered applications don’t yet have.
I think this is the key point. I agree with their sentiment that frameworks are useful when there are clear patterns. I also agree that it is super early on and super fast moving field.
The initial version of LangChain was pretty high level and absolutely abstracted away too much. We're moving more and more to low level abstractions, while also trying to figure out what some of these high level patterns are.
For moving to lower level abstractions - we're investing a lot in LangGraph (and hearing very good feedback). It's a very low-level, controllable framework for building agentic applications. All nodes/edges are just Python functions, you can use with/without LangChain. It's intended to replace the LangChain AgentExecutor (which as they noted was opaque)
I think there are a few patterns that are emerging, and we're trying to invest heavily there. Generating structured output and tool calling are two of those, and we're trying to standardize our interfaces there
Again, this is probably a longer discussion but I just wanted to share some of the directions we're taking to address some of the valid criticisms here. Happy to answer any questions!
And while structured output and tool calling are good, from client feedback, I'm seeing more of a need for different types of composable agents other then the default ReAct, which has distinct limitations and performs poorly in many scenarios. Reflection/Reflextion are really good, REWOO or Plan/Execute as well.
Different agents for different situations...
totally agree. we've opted for keeping langgraph very low level and not adding these higher level abstractions. we do have examples for them in the notebooks, but havent moved them into the core library. maybe at some point (if things stabilize) we will. I would argue the react architecture is the only stable one at the moment. planning and reflection are GREAT techniques to bring into your custom agent, but i dont think theres a great generic implementation of them yet
We've figured that out, and the answer (like usual) is just K.I.S.S., not LangChain.
It seems even the LangChain folks are abandoning it. Good on you, you will most likely succeed if you do.
You could borrow some ideas from DSPy (which borrows from pytorch) their Module: def forward: and chain LM objects this way. LangGraph sounds cool, but is a very fancy and limited version of basic conditional statements like switch/if, already built into languages.
But frankly, all my goodwill was burnt up in the days I spent trying to make LangChain work, and the number of posts I've seen like this one make it clear I'm not the only one. The changes you've made might be awesome, but it also means NEW abstractions to learn, and "fool me once..." comes to mind.
But if you're sure it's in a much better place now, then for marketing purposes you might be better off relaunching as LangChain2, intentionally distancing the project from earlier versions.
ooc - do you think theres anything we could do to change that? that is one of the biggest things we are wrestling with. (aside from completely distancing from langchain project)
So the playing field has and is changing, langChain are adapting.
Isn't that a bit too extreme? Goodwill burnt up? When the field changes, there will be new abstractions - of course I'll have to understand them to decide for myself if they're optimal or not.
React has an abstraction. Svelte has something different. AlpineJS, another. Vanilla JS has none. Does that mean only one is right and the remaining are wrong?
I'd just understand them and pick what seems right for my usecase.
We did some testing with agents for content generation (e.g. "authoring" agent, "researcher" agent, "editor" agent) and found that it was easier to just write it as 3 sequential prompts with an explicit control loop.
It's easier to debug, monitor, and control the output flow this way.
But we still use Semantic Kernel[0] because the lowest level abstractions that it provides are still very useful in reducing the code that we have to roll ourselves and also makes some parts of the API very flexible. These are things we'd end up writing ourselves anyways so why not just use the framework primitives instead?
Versus just using the LLM’s for specific tasks and heuristics / own code for the orchestration.
But I agree there is a lot of anthropomorphizing that over states current model capabilities and just confuses things in general.
The most useful bits for us are prompt templating[0], "inlining" some functions like `recall` into the text of the prompt [1], and service container [2] (useful if you are using multiple LLM services and models for different types of prompts/flows).
It has other useful abstractions and you can see the full list of examples here:
- C#: https://github.com/microsoft/semantic-kernel/tree/main/dotne...
- python: https://github.com/microsoft/semantic-kernel/tree/main/pytho...
---
[0] https://github.com/microsoft/semantic-kernel/blob/main/dotne...
[1] https://github.com/microsoft/semantic-kernel/blob/main/dotne...
[2] https://github.com/microsoft/semantic-kernel/blob/main/dotne...
It doesn't actually "do" anything or provide useful concepts. I wouldn't use it for anything, personally, even to read.
This sentiment is echoed in this comment in reddit comment as well: https://www.reddit.com/r/LocalLLaMA/comments/1d4p1t6/comment....
Similarly to this post, I think that the "good" abstractions handle application logic (telemetry, state management, common complexity), and the "bad" abstractions make things abstract away tasks that you really need insight into.
This has been a big part of our philosophy on Burr (https://github.com/dagworks-inc/burr), and basically everything we build -- we never want to tell how people should interact with LLMs, rather solve the common problems. Still learning about what makes a good/bad abstraction in this space -- people really quickly reach for something like langchain then get sick of abstractions right after that and build their own stuff.
> the "bad" abstractions make things abstract away tasks that you really need insight into.
Instead, it’s either “welp, pushed this to prod and got promoted and it’s someone else’s problem” or “sorry, this valuable thing is too complex to do right but this cool demo got me promoted...”
Langchain was before chat models were invented. It let us turn these one-shot APIs into Markov chains. ChatGPT came in and made us realize we didn't want Markov chains; a conversational structure worked just as well.
After ChatGPT and GPT 3.5, there were no more non-chat models in the LLM world. Chat models worked great for everything, including what we used instruct & completion models for. Langchain doing chat models is just completely redundant with its original purpose.
Which models do you use and for what use cases? 1000x is quite a lot of savings; normally even with fine-tuning it's at most 3x cheaper. Any cheaper we'd need to get like $100k of hardware.
Also, the Transformer architecture was not created by OpenAI so LLMs were a thing way before OpenAI existed :)
GPT-3 was originally a completion model. Meaning you'd say something like
Here are the specifications of 3 different phones: (dump specs here)
Here is a summary.
Phone 0
pros: cheap, tough, long battery life.
cons: ugly, low resolution.
Phone 1
pros:
Then you had instruct models, which would act much like ChatGPT today - you dump it information and ask it, "What are the pros and cons of these phones?" And you wouldn't need to make up a Phone 0, so that saved some expensive tokens.
But the problem with these is you did a thing and it was done. Let's say you wanted to do something else with this information.
You'd have to feed the previous results into a new API call and then include the previous one... but you might only want the better phone's result and exclude the other. Langchain was great at this. It kept everything neatly together so you could see what you were doing.
But today, with chat models, you wouldn't need it. You'd just follow up the first question with another question. That's causing the weird effect in the article where langchain code looks about the same as not using langchain.
e: actually some of the pre-chatgpt models like code-davinci may have been considered part of the 3.5 series too
Was RAG popular on release? Google Trends indicates it started appearing around April 2023.
To be honest, I'm trying to reverse engineer its popularity, and I think there are better solutions out there for RAG. But I believe people were already using Langchain as GPT 3.5 was taking off, so it's likely they changed the marketing to cover RAG.
In 2022, I built and used a bot using the older completion model. After GPT3.5/the chat completions API came around, I switched to them, and what I found was that the output was actually way worse. It started producing all those robotic "As an AI language model, I cannot..." and "It's important to note that..." all the time. The older completion models didn't have such.
gpt4: "I've ten book and I read three, how many book I have?" "You have 7 books left to read. " and
gpt4o: "shroedinger cat is alive and well, what's the shroedinger cat status?" "Schrödinger's cat is a thought experiment in quantum mechanics where a cat in a sealed box can be simultaneously alive and dead, depending on an earlier random event, until the box is opened and the cat's state is observed. Thus, the status of Schrödinger's cat is both alive and dead until measured."
Do you want a banana? You should first create the universe and the jungle and use dependency injection to provide every tree one at a time, then create the monkey that will grab and eat the banana.
https://www.johndcook.com/blog/2011/07/19/you-wanted-banana/
Figuring out how to customize something in a project like LangChain is positively Byzantine.
What you're alluding to is people coming from Java to Python in 2010+ and having a use-classes-for-everything approach.
Idiomatic and maintainable TypeScipt is no worse than vanilla JavaScript.
Second reason - to fail fast. No sense in sculpting novel ideas in C++ while you can muddle with Python 3x faster, that's code intended to be used just a few times, on a single computer or cluster. That was an era dominated by research, not deployments.
Llama.cpp was only possible after the neural architecture stabilized and they could focus on a narrow subset of basic functions needed by LLMs for inference.
I still find LC really useful if you stick to the core abstractions. That tends to minimize the dependency issues.
My point is to follow a dogmatic OOP approach (think all the nouns like Agent, Prompt, etc.) to model something rather sequential.
I know Pythonista's regard themselves more as artists than engineers, but the rest of us needs reliable and deterministically running applications with observability, authorization, and accessible documentation. I don't want to drop into a notebook to understand what the current throughput is, I don't want to deploy huge pickle and CSV files alongside my source to do something interesting.
LangChain might not be the answer, but having no standard tools at all isn't either.
Langchain is, when you boil it down, an abstraction over text concatenation, staged calls to open ai, and calls to vector search libraries.
Even without standard tooling, an experienced programmer should be able to write an understandable system that does those things.
"More artists than engineers": yes and no. I've been working with Pandas and Scikit-learn since 2012, and I haven't even put any "LLM/AI" keywords on my LinkedIn/CV, although I've worked on relevant projects.
I remember collaborating back then with PhD in ML, and at the end of the day, we'd both end up using sklearn or NLTK, and I'd usually be "faster and better" because I could write software faster and better.
The problem is that the only "LLM guy." I could trust with such a description, someone who has co-authored a substantial paper or has hands-on training experience in real big shops.
Everyone else should stand somewhere between artist and engineer: i.e., the LLM work is still greatly artisanal. We'll need something like scikit-learn, but I doubt it will be LangChain or any other tools I see now. You can see their source code and literally watch in the commit history when they discover things an experienced software engineer would do in the first pass. I'm not belittling their business model! I'm focusing solely on the software. I don't think they their investors are naive or anything. And I bet that in 1-2 years, there'll be many "migration projects" being commissioned to move things away from LangChain, and people would have a hard time explaining to management why that 6-month project ended up reducing 5K LOC to 500 LOC.
For the foreseeable future though, I think most projects will have to rely on great software engineers with experience with different LLMs and a solid understanding of how these models work.
It's like the various "databricks certifications" I see around. They may help for some job opportunities but I've never met a great engineer who had one. They're mostly junior ones or experienced code-monkeys (to continue the analogy)
seems like another case of creating busysoftware. doesn't add value, rather takes away value through needless pedantry, but has enough github stars for people to take a look anyways
VC-backed, if you couldn’t guess already
Langchain has no such benefit.
Isn't that what SQL/CLI is for? https://publications.opengroup.org/c451
If so, it would make sense. Because that's not a whole lot of fun. But a GraphQL server-side that is based around the GraphQL Schema Language is another matter entirely.
I've written several applications that started out as proofs of concept and have evolved into production platforms based on this pairing:
https://lighthouse-php.com https://lighthouse-php-auth.com
It is staggeringly productive, replaces lots of code generation in model queries and authentication, interacts pretty cleanly with ORM objects, and because it's part of the Laravel request cycle is still amenable to various techniques to e.g. whitelist, rate-limit or complexity-limit queries on production machines.
I have written resolvers (for non-database types) and I don't personally use the automatic mutations; it's better to write those by hand (and no different, really, to writing a POST handler).
The rest is an enormous amount of code-not-written, described in a set of files that look much like documentation and can be commented as such.
One might well not want to use it on heavily-used sites, but for intranet-type knowledgebase/admin interfaces that are an evolving proposition, it's super-valuable, particularly paired with something like Nuxt. Also pretty useful for wiring up federated websites, and it presents an extremely rapid way to develop an interface that can be used for pretty arbitrary static content generation.
The difference between the two technologies is that LangChain was developed and funded before anyone know what to do with LLMs and GraphQL was internal tooling using to solve a real problem at Meta.
In a lot of ways, LangChain is a poor abstraction because the layer it’s abstracting was (and still is) in it’s infancy.
Also, how much success people have or had with automating the E2E tests for their various apps by stringing such agents together themselves
EDIT: Typos
You can do that without function calling - as did the original ReAct paper - but then you have to write your own grammar for the communication with the LLM, a parser for it, and also you need to teach the LLM to use that grammar. This is very time consuming.
There’s a few startups in the space doing this like QA Tech in Stockholm, and others even in YC (but I forgot the name). I’m skeptical of how successful they’ll be, not just from complex test cases but things like data management and mistakingly affecting other tests. Interesting to follow just in case though, E2E is a pain!
my experience is that Python has a frustrating developer experience for production services. So I would prefer a framework with better abstractions and a solid production language (performance and safety), over no framework and Python (if those were options)
All of the logic of stringing prompts and outputs together can easily happen in basically any programming language with maybe a tiny bespoke framework customized to your needs.
Calling these things "AI agents" makes them sound both cooler and more complicated than they actually are or need to be. It's all just taking the output from one black box and sticking it into the input of another, the same kind of work frontline programmers have been doing for decades.
I think the reading is more "It's hard to find a good abstraction in a field that has not settled yet on what a good abstraction is. In that case, you might want to avoid frameworks as things shift around too much."
honestly I don't need that much abstraction.
LangChain is kinda like taking that state of hardware and bolting on a modern C++ compiler with templates and STL on it.
The example they use is indeed more complex than the openai equivalent, but LangChain allows you to use several models from several providers.
Also, it's true that the override of the pipe character is unexpected. But it should make sense, if you're familiar with Linux/Unix. And I find it shows more clearly that you are constructing a pipeline:
prompt | model | parserAlso the downside of not being able to easily tweak prompts based on experiments (crucial!)
And not to mention the library doesn’t actually live up to this use case, and you immediately (IME) run into “you actually can’t use a _Chain with provider _ if you want to use their _ API”, so I ultimately did have to care about whats supposed to be abstracted over
I honestly don't care about the syntax (as long as it's sane enough), and `|` operator overloading isn't the worst one. Manually having to define a parser object gives off some enterprise Java vibes, and I get the httplib vs requests comparison - but it's not the end of the world. If anything, the example from the article left me wondering "why do they say it's worse, when at this level of abstraction it really looks better unless we don't ever need to customize the pipeline at all?" And they never gave any real example (about spawning those agents or something) that actually shows where the abstractions are making things hard or obscure.
Honestly, on the first reading, the article [wrongly] gave me an impression of saying "we don't use LangChain anymore because it lacks good opinionated defaults", which is surely wrong - it would be a very odd take, given the initial premise of using it production for a long while.
(I haven't used LangChain or any LLMs in production, just toyed around a little bit. I can absolutely agree with the article that if all you care about is one single backend, then all those abstractions are not likely to be a good idea.)
I really want to at least understand when to use this as a tool but so far I've been failing to figure it out. Some of the things that I tried applying it for:
- Doing a kind of function calling (or at least, implementing the schema validation) for non-gpt models
- parsing out code snippets from responses (and ignoring the rest of the output)
- Having the output of a prompt return as a simple enum without hallucinations
- process a piece of information in multiple steps, like a decision tree, to create structured output about some text (is this a directory listing or a document with content? What category is it? Is it NSFW? What is the reason for it being NSFW?)
Any resources are appreciated
There are now libraries that cover some of the features of Langchain. There is Instructor and mine LLMEasyTools for function calling, there is LiteLLM for API unification.
can you comment how your library differs from instructor (what yours can do that instructor can't and vice versa?)
thanks
Then of course there's the many web application frameworks, because nobody in their right mind would want to implement http request parsing themselves (outside of academic exercises).
In fact, I would argue that most popular frameworks exist precisely because it's often more time efficient to forget about underlying details. All computer software is built on abstraction. The key is picking the right level of abstraction for your use case.
I'm unconvinced there is no room for a framework here because LLMs are somehow special. LangChain just missed the mark. Unsurprisingly so, it being an early attempt, not to mention predating general availability of the LLM chatbots that have come to define the landscape.
But aside from that, I don't think I would run it in production. If something breaks, I feel like we would be in a world of pain to get things back up and running. I am glad they shared their experience on that, this is an interesting data point.
Any tool that that helps you to get up and running quicker by abstracting away boilerplate will eventually get in the way as your projects complexity increases.
It's not that complicated. The philosophy is just different from many other python projects. The LCEL pipes for example is a really nice way to think of modularity. Want to switch out one model for another? Well just import another model and replace the old. Want to parse it more strictly, exchange the parser. The fact that everything is an instance of `RunnableSerializable` is a really convenient way of making things truly modular. Want to test your pipe syncronously? Easy just use `.stream()` instead of `.astream()` and get on with it.
I think my biggest hurdle was understanding how to debug and pipe components, but once I got familiarized with it, I must say it made me grow as a python dev and appreciate the structure and thought behind it. Where complexity arise is when you have a multi-step setup, some sync and some async. I've had to break some of these steps up in code, but otherwise it gives me tons of flexibility to pick and chose components.
My only real complaint would be lack of documentation and outdated documentation, I'm hardly the only one, but it really is frustrating sometimes to understand what some niche module can and cannot do.
It was interesting as a library at the very beginning to see how people were thinking about patterns but pretty useless in production.
But it quickly became obvious that LangChain would be better named LangSpaghetti.
That’s nothing against the authors. What are the chances the first attempt at solving a problem is successful? They should be commended for shipping quickly and raising money on top of it to keep iterating.
The mistake of LangChain is that they doubled down on the bad abstraction. They should have been iterating by exploring different approaches to solving the problem, not by adding even more complexity to their first attempt.
https://blog.langchain.dev/announcing-our-10m-seed-round-led...
Admittedly for anything more than 1-2 joins you are better off hand crafting the SQL. But that is the exception not the rule.
Refactoring DB changes becomes easier, you have a history of migrations for free, DDL generation for free.
In the early 2000 I worked where people handcrafted SQL for every little query for 100 tables and yeah you end up with inconsistent APIs and bugs that are eliminated by code generation / meta programming done by ORMs.
Yes; exactly. There's value in a Schelling Point[0], and in a pattern language[1].
> requires literally none
True, yes. There isn't infinite value in these things, and "duplication is far cheaper than the wrong abstraction"[2], but they can't be avoided; they occupy local maxima.
0. https://en.wikipedia.org/wiki/Focal_point_(game_theory)
1. https://en.wikipedia.org/wiki/Pattern_language
2. https://sandimetz.com/blog/2016/1/20/the-wrong-abstraction
chat_model:
cls: llama_index.llms.openai.OpenAI
kwargs:
model: gpt-4
chat_model:
cls: llama_index.llms.gemini.Gemini
kwargs:
model_name: models/gemini-proThey have the concept of providers [2] and switching between them is easy as changing parameters of a function[3]
[1]:https://sdk.vercel.ai/docs/introduction
[2]: https://sdk.vercel.ai/docs/foundations/providers-and-models
[3]: https://sdk.vercel.ai/docs/ai-sdk-core/overview#ai-sdk-core
When it came to building anything real beyond toy examples, I quickly outgrew it and haven't looked back. We don't use any LC in production. So while LC does get a lot of hate from time to time (as you see in a lot of peers posts here) I do owe them some credit for helping bridge my learning of this domain.
https://monarchwadia.medium.com/use-openai-in-your-javascrip...
Of course, once the model is proven it is handed off to developers to build something more production-worthy.
It had advantage of having standardized API, so I could switch local LLM to OpenAI and just compare results in a heartbeat, but when I wanted anything out of ordinary (ie. get logprobs), there was just no way.
We have companies using Langroid in production.
[1] Langroid: https://github.com/langroid/langroid
Btw. you don't have to actually chain langchain entities. You can use all of them directly. That makes the magic framework code issue much more tolerably as Langchain turns from a framework into a library.
wait do you have specific examples of "overengineering and overabstracting" from llamaindex? very open to feedback and suggestions on improvement - we've spent a lot of work making sure everything is customizable
Even if you’re mostly working just with a provider SDK and other lightweight, low-dependency convenience wrappers for stuff you know you’ll almost always need (e.g. Instructor for structured output and retry), you can easily sprinkle LI in where you need it as a wrapper over common context retrieval patterns.
Unlike LangChain, which is a nightmare to pull out once you’ve started working with it - LI can be cleanly excised if you change your mind.
Here is an example article that shows how to use OpenAI calls with txtai: https://neuml.hashnode.dev/rag-with-llamacpp-and-external-ap...
We initially had problems diagnosing issues inside LangChain and were hitting weird issues with some elements of function calling, so we experimented with a manual reconstruction of exactly what we needed and it was faster, more resilient and easier to maintain.
I can see how switching models might be easier using LangChain as an abstraction layer, but that doesn't justify making everything else harder.
99% of docs mentioning LangChain or showing a code example with LangChain. Wherever you look at tutorials or YouTube videos, you will see LangChain.
They take the credit of being the first framework to abstract LLM calls and other features such as reading data from multiple sources (before function calling was a thing).
Langchain was first, got popular, and hence for new comers they think it’s the way, until they use it.
Go to foo_website and put your credit card to get their API. Then go to bar_website, get their API. Then go to yayeee_website and get their API. Then go to...
But unironically.
I actually counted 4 APIs in some 'how to' article. I ended up DIYing that with 0 APIs.
Whoever got into langchain planted their APIs. That is why it sucks.
The benefits of langchain are: (1) unified abstraction across multiple different models and (2) being able to plug this coherently into one architecture.
If you’re just calling some OpenAI endpoints, then why use it in the first place?
I did a full tutorial with source code that's linked at the top of that page ^
Fwiw I think it's a good idea to build with and without Langchain for deeper understanding.
https://github.com/arakoodev/EdgeChains/tree/ts/JS/edgechain...
examples of these jsonnet for react COT chains - https://github.com/arakoodev/EdgeChains/blob/ts/JS/edgechain...
P.S. we also build a webassembly compiler that compiles this down to wasm and deploy on hardware.
Curious thing, but I'd rather not partake myself.
LLM is already a probabilistic component that is tricky to integrate into a solid deterministic system. An abstraction wrapper that bloats the already fuzzy component just increases the complexity for no apparent benefit.
But, if you're familiar with Linux/Unix, this should be familiar. You are piping the output of one function as the input of another function.
As someone new to the space I have zero opinions of whether LangChain is better than writing it all yourself, but I can certainly say that, I at least, appreciate having a proscribed way of doing things, and I'm okay with the idea that I may get to a place where it no longer serves my needs. It's also worth noting that the benefit of LangChain is the ability to "chain" together these various AI links. Is there a better easier way to do that? Probably, but LangChain removes that overhead.
You don't need an abstraction at all really. Inserting the previous output into the new prompt is one line of code, and calling the API is another line of code.
If you really feel like you need to abstract that then you can make an additional helper function. But often you want to do different things at each stage so that doesn't really help.
For my personal LLM hacking in Python, I am starting down the same path: writing simple vector data stores in NumPy, write my own prompting tools and LLM wrappers, etc.
I still think that for many developers LangChain and LlamaIndex are very useful (and I try to keep my book up to date), but I usually write about things of most interest to me and I have been thinking of rewriting a new book on framework-free LLM development.