Let's be honest: Lisp probably won't ever get bigger than Python, unless Python for whatever reason starts dying on its own. But if AI ever gets serious about interpretability, formal reasoning, program synthesis - all the stuff Lisp was built for - it just might quietly become relevant again in research contexts, without ever reclaiming mainstream status.
Scicloj has been building out a serious ML stack in Clojure - noj, metamorph.ml, scicloj.ml.tribuo, libpython-clj for Python interop. Beside that, people been proving that 'code is data' is exactly what makes it a better target for LLMs. Clojure is most token efficient PL - it's been proven. There are some recent interesting clj projects in relevance:
—
LLMs already internally operate on embeddings - dense floating-point vectors in high-dimensional space - they don't need "intermediary language" - there's no discrete symbolic reasoning - it's continuous numerical computation all the way through.
If you made an LLM to generate bytecode, you'd need it to "read" bytecode tokens as context for predicting the next bytecode token. The model would need to have learned the statistical patterns of bytecode sequences, which are far less structured and too arbitrary than (human readable) source code. Bytecode is optimized for machine execution, not for having patterns for a next-token predictor.
So Lisp actually is the sweet spot in the direction you're gesturing at, but you got the direction backwards. You don't want to go lower (toward bytecode), you want to go toward representations that have: fewer syntactic rules, structural, and high in local predictability. That is basically Lisp. The AST is the syntax. There's no parsing ambiguity, minimal syntactic variation, and the structure is self-describing. The model spends fewer tokens on syntactic ceremony and more on semantic content. That is why LLMs are surprisingly great at generating Elisp and Clojure code.
SBCL is a fine choice for a solo project or research; recommending it over Clojure for a team/production setting though requires ignoring most of what actually matters. Every language stack choice is a bet on where the tradeoff between power and pragmatism pays off for your specific context. Clojure's bet is a more conservative than CLs, yet both may seem (for some) more aggressive bets than just using Python. It all depends on the team.
