To be clear, this seems like a cool project and I dont want to be too negative about it, but i just think this was an entirely foreseeable outcome, and the amount of people excited about this JIT project when it was announced shows how poorly a lot of people understand what goes into making a language fast.
It was not universal. But it was very common and at least plausibly a majority view, so this idea wasn't just some tiny minority view either.
I consider this idea falsified now, pending someone actually coming up with a JIT/compiler/whatever that achieves this goal. We've poured millions upon millions of dollars into the task and the scripting languages still are not as fast as C or static languages in general. These millions were not wasted; there were real speedups worth having, even if they are somewhat hard on RAM. But they have clearly plateaued well below "C speed" and there is currently no realistic chance of that happening anytime soon.
Some people still have not noticed that the idea has been falsified and I even occasionally run into someone who thinks Javascript actually is as fast as C in general usage. But it's not and it's not going to be.
To be very pedantic, the problem is not that these are dynamic languages _per se_, but that they were designed with semantics unconcerned with performance. As such, retrofitting performance can be extremely challenging.
As a counterexample of fast and dynamic: https://julialang.org/ (of course, you pay the prize in other places)
I agree with your comment overall, though.
JITs are really only ideal for request-processing systems, in which a) memory is abundant b) the same code paths run over and over and over again, and c) good p99 latency is usually the bar.
In contrast, in user facing apps, you usually find that a) memory is constrained b) lots of code runs rarely or in some cases once (e.g. the whole start-up path) c) what would be considered good p99 latency for a server can translate to pretty bad levels of jank.
JITs can't do anything if the code you care about runs rarely and causes a frame skip every time you hit it, either because the JIT hasn't triggered yet due to too-few samples, or the generated code has been evicted from the JIT cache because you don't have memory to spare. And if you have code that needs to run fast _every_ time it runs, the easiest way to do that is to start with fast code already compiled and ready to execute.
We saw this play out when android moved from Dalvik (JIT) to ART (AoT compilation). Apple figured this out years earlier.
Of course it's not that there are no highly performant apps built on JIT runtimes. But it's a significant headwind.
(Most of the above applies equally to tracing GC, btw)
All those languages are just as dynamic as Python, more so given the dynamically loading of code with image systems, across network, with break into debugger/condition points and redo workflows.
Something else is going on.
Is this in the article? I don't see Python's semantics mentioned anywhere as a symptom (but I only skimmed).
> shows how poorly a lot of people understand what goes into making a language fast.
...I'm sorry but are you sure you're not one of these people? Some facts:
1. JS is just as dynamic and spaghetti as Python and I hope we're all aware that it has some of the best jits out there;
2. Conversely, C++ has many "optimizing compiler[s]" and they're not all magically great by virtue of compiling a statically typed, rigid language like C++.
I think if I was being paid to make CPython faster I’d spend at least a year changing how objects work internally. The object model innards are simply too heavy as it stands. Therefore, eliminating the kinds of overheads that JITs eliminate (the opcode dispatch, mainly) won’t help since that isn’t the thing the CPU spends much time on when running CPython (or so I would bet).
But I'm trying to find/recall a blog post that detailed the different steps in shrinking the CPython object struct...
If you say that's not enough, more radical changes needed, I would understand.
Given any arbitrarily optimized thing, it is always possible to optimize it more. And the fact that it's possible to optimize it more is not meant as a criticism of folks who did the previous optimizations.
So, I have no doubt that Mark and others have worked on exactly the thing I'm talking about and that they've gotten wins. And I have no doubt that more can be done. Also, not saying I would do a better job at it than Mark or anyone else
I think that the FFI makes it super hard to do most of the optimizations I'd want to do. Maybe it makes them impossible even. The game is to find any chance for size reduction and fast path simplification that doesn't upset FFI
- Most of the work has just been plumbing. Int/float unboxing, smarter register allocation, free-threaded safety land in 3.15+.
- Most JIT optimizations are currently off by default or only triggers after a few thousand hits, and skips any byte-codes that look risky (profiling hooks, rare ops, etc.).
I really recommend this talk with one of the Microsoft faster Cpython developers for more details, https://www.youtube.com/watch?v=abNY_RcO-BU
> - Most of the work has just been plumbing. Int/float unboxing, smarter register allocation, free-threaded safety land in 3.15+.
The first part is true, but for the second sentence: none of that is guaranteed to land in 3.15+. We proposed to land them, that doesn't mean they will. Landing a PR in CPython is subject to maintainer time and reviewer approval, which doesn't always happen. I proposed a few optimizations for 3.14 that never landed.
> Most JIT optimizations are currently off by default or only triggers after a few thousand hits
It is indeed true we only trigger after a few thousand hits, but all optimizations that we currently have are always enabled. We don't sandbag the JIT on purpose.
It's easy to dismiss our efforts, but Ruby is just as dynamic if not more than Python. It's also a very difficult language to optimize. I think we could have done the same for Python. In fact the Python JIT people reached out to me when they were starting this project. They probably felt encouraged seeing our success. However they decided to ignore my advice and go with their own unproven approach.
This is probably going to be an unpopular take but building a good JIT compiler is hard and leadership matters. I started the YJIT project with 10+ years of JIT compiler experience and a team of skilled engineers, whereas AFAIK the Python JIT project was lead by a student. It was an uphill battle getting YJIT to work well at first. We needed grit and I pushed for a very data-driven approach so we could learn from our early failures and make informed decisions. Make of that what you will.
Yes Python is hard to optimize. I Still believe that a good JIT for CPython is very possible but it needs to be done right. Hire me if you want that done :)
Several talks about YJIT on YouTube for those who want to know more: https://youtu.be/X0JRhh8w_4I
Everything is a message, the meta classes that define object shapes can change any time some feels like it, there are methods like becomes: that completely replaces an object across all its references on the running image, break into debugger and redo after whatever was changed while into the debugger, code loaded over network,....
As if PyPy and GraalPy didn't even exist as well.
Given this "it's dicts all the way down" nature of CPython, I'm curious if the recent hash table theoretical breakthrough[1] discussed here[2] a few months ago may eventually help making it much faster, given the compounding of dict upon dict?
[1] https://www.quantamagazine.org/undergraduate-upends-a-40-yea...
I haven't checked, but I wouldn't be surprised if more Python versions contained breaking changes than not.
Smalltalk, Self, Lisp, are highly dynamic, their JIT research are the genesis of modern JIT engines.
For some strange reason, Python community rather learns C, calls it "Python", instead of focusing why languages that are just as dynamic, have managed already a few decades ago.
Hard to put a finger on what exactly, but Python has never been so interested in purity, rather in pragmatic functionality and it ends up in a place where it gives access to C style idioms and API, see for example the os module.
Maybe it is that Python doesn't have it's own model of the world, but it provides a dynamic language facade to the C model of the world.
Seems like the development was funded by Shopify and they got a ~20% performance improvement. https://shopify.engineering/ruby-yjit-is-production-ready
A similar experience in the Python community is that Microsoft funded "Faster CPython" and they made Python 20-40% faster.
At this point it's a great didactic tool and a passion project surely? Or, has advantages in other dimensions like runtime size, debugging, and .pyc coverage, or in thread safe code or ...
Unoptimised jit < optimised interpreter (at least in this instance)
They are working on it presumably because they think there will eventually be a speed ups in general or at least for certain popular workloads.
Still, to directly answer the first question, I would hope even if there wasn't obvious performance improvements immediately, if folks want to work on this, I see no reason not to explore it. If we are lucky, we find improvements we didn't expect.
Adding more optimizations improves things from there.
But the point is, a JIT can be a speedup just because it isn’t an interpreter (it doesn’t dynamically dispatch ops).
Everyone knows Python is hard to optimize, that's why Mojo also gave up on generality. These claimed 20-30% speedups, apparently made by one of the chief liars who canceled Tim Peters, are not worth it. Please leave Python alone.
I don't remember the Faster CPython Team claiming JIT with a >50% speedup should have happened two years ago, can you provide a source?
I do remember Mark Shannon proposed an aggressive timeline for improving performance, but I don't remember him attributing it to a JIT, and also the Faster CPython Team didn't exist when that was proposed.
> apparently made by one of the chief liars who canceled Tim Peters
Tim Peters still regularly posts on DPO so calling him "cancelled" is a choice: https://discuss.python.org/u/tim.one/activity.
Also, I really can not think who you would be referring to as part of the Faster CPython Team, of which all the former members I am aware of largely stayed out of the discussions on DPO.
