But considering that optimized scalar code performance has moved, what, maybe 40% over the last two decades, I'm going to say "not much". Compilers are sexy, but they're very much a solved problem. If we were all forced to get by with the optimized performance we saw from GCC 2.7.2, I think we'd all survive. Most of us wouldn't even notice the change.
But in the context of JIT compilers for dynamically typed languages, in particular the space involving runtime inferred hidden type models, there is a TON of work left on the table.
It hasn't been paid much attention to in academia, IMHO largley because of a historical perspective on optimizing dynamic languages as "not classy" work among language theorists. I hope that perspective changes over time.
Not for all of the other widely-used languages that still have incredibly simple interpreters. Think how much energy could have been saved if Ruby, Python, and PHP were all as fast as your average JS engine.
Compilers are sexy, but they're very much a solved problem.
For sequential languages the problem has shifted: how can I get a performant compiler easily. The most interesting answer to this question is PyPy's meta-tracing, and that's work is from 2009, and far from played out.
A 40% decrease in optimization is enough to drop framerates from 60fps to 30fps easily, so I'm pretty sure we would notice it.
I'm not convinced. Raw single-thread number crunching performance is somewhere around _two to three fold_, clock-for-clock, on Intel x86, over that of 10-15 years ago. What methodology do you use to attribute only a fraction of those gains to language optimizers? And even if you are correct, why is it meaningful? Who is going to have invested energy in optimising the shit out of mundane codegen when hardware performance will have just come and stolen your thunder a few months later?
The problem we have now is that CPUs are gaining ever more complex behaviour, peculiarities, and sensitivities. I'd say compiler engineering is far from a "solved problem", even for statically-typed languages.
No they're not, and won't be for long (ever?). However it does not matter because they are "good enough".
Compilers are driven by heuristics which provide "reasonable" results in most cases for common architectures. But they still leave a lot on the table. Compiler writers have to trade compile-time with execution-time. Now we're not talking about an order of magnitude, but rather ~20%-30% in some workloads. When it matters (I guess for people like Google/Facebook/Amazon/... it translates in electricity bill and a number of racks to add to the datacenter) people may have to get down to the assembly level for a very small (and hot) part of the program.
Python, Ruby, PHP, and Perl aren't "the rest of the world"? As far as compilers are concerned, all of those languages have more troublesome semantics than JavaScript does.
> And only the web world is so obssessed with smart compilers compensating (impressively, but still far from being sufficient) for multiple deficiencies in the language design.
You have no idea how much compilers have to compensate for the deficiencies in C and C++'s design.
Which ones are these? All of the statically-typed and well-designed languages I can think of are, at the least, hard to compile well, if not hard to compile in the first place. (Haskell and Rust both come to mind; there are few Haskell compilers other than GHC, and no Rust compilers other than rustc.) The languages that are easier to compile are either not statically-typed in a useful way, or not well-designed, or both.
Also, many of the improvements done for JS will certainly trickle down to Python, Ruby, PHP eventually.
It's also encouraging to see them opening up about future directions rather than just popping full-blown features from the head of Zeus every so often. (Not that they owe us anything... ;-)
(Edit: it's also damned impressive for 2 people in 3-4 months.)
But as soon as we realized that we had such a huge compile time opportunity, of course we optimized the heck out of the new compiler for the kinds of things that we always wished LLVM could do - like very lightweight patchpoints and some opcodes that are an obvious nod for what dynamic languages want (ChillDiv, ChillMod, CheckAdd, CheckSub, CheckMul, etc).
Is this a knock on LLVM then?
I wonder then specifically if this brings to light any concerns over Swift (another dynamic language, and was created by the same person who created LLVM as well). [2]
Seems weird that the original creator of LLVM was able to make a dynamic language such as Swift - without any problems.
> While LLVM is an excellent optimizer, it isn’t specifically designed for the optimization challenges of dynamic languages like JavaScript.
LLVM was, more-or-less, designed around C++ semantics. B3 is not going to be a better C++ compiler than LLVM. LLVM is not going to be a better JS compiler than B3. They're taking different evolutionary paths.
Also its compilation speed is supposed to be fast in terms of AOT compilation, but not necessarily fast enough for JITing javascript in web-pages.
VxWorks is great at driving martian rovers, but would be horrible at running Adobe Photoshop or Maya.
Why does everything either need to be a snub or a boon?
LLVM is great at generating compiled binaries - it just takes resources (time + memory) to do so - in practical terms, the compilation time only affects the programmers and developers running multiple compiles over a work day, working on the codebase itself. For consumers who are running the programs, LLVM makes great optimized binaries.
In a JS engine, the compiler doesn't have access to the source code until a user loads up the page, so it's in essence doing "compilation" on the spot. And in this case compilation memory and time usage matters.
Swift itself is a "compiled" lanugage - meaning that in development, a developer needs to hit that "compile" button in XCode to generate a binary file which is then distributed and run. In this case, LLVM can use all cores on that big honking Mac Pro or shiny MacBook Pro 15" retina all it wants - take all 16 cpus and 32 gigs of ram and crank it and make a nice binary that comes out optimized to run on a iPad Air.
Of course things like Java conflates the two styles (source code compiled to bytecode and then to run on the JVM) - but in the Java ecosystem there are significant optimizations in both the compile phase (ie javac helloworld.java) as well as ahead of time compilation in the JVM itself.
If you're working on a JIT, this matters. A lot. Webkit isn't the first project has has used or worked with LLVM in this manner and abandoned it. PyPy has also investigated using LLVM and stopped doing that.
LLVM isn't bad, it just isn't good at solving this problem it wasn't designed for. If anything, I think it's remarkable LLVM has worked out as well for Webkit as it did.
The conclusion is that it's mostly dead or worse, because of the performance hit when typechecked code calls untyped code.
(And also because of the details of Typed Racket, which perfectly reasonably translates into ordinary untyped Racket, but then that is then mangled by GNU lightning (!).)
But...
"At the same time, we are also challenged to explore how our evaluation method can be adapted to the world of micro-level gradual typing, where programmers can equip even the smallest expression with a type annotation and leave the surrounding context untouched. We conjecture that annotating complete functions or classes is an appropriate starting point for such an adaptation experiment."
I think we don't have great tools for helping with this sort of optimization. One can use perf to find cache misses, but that doesn't necessarily tell the whole story, as you might blame some other piece of code for causing a miss. Maybe I should try cachegrind again...
This is why, when they compare it to v8/etc, it's kind of funny. They all have the same curve.
Basically all of these things, all of them, end up with roughly the same deficiencies once you cherry pick the low hanging fruit[1], and then they stall out, and get replaced a few years later when someone decides thing X can't do the job, and they need to write a new one. None of them ever get to a truly good state.
Rinse, wash, repeat.
The only thing these things make real progress, is by doing what LLVM did - someone works on it for years.
Let me quote a former colleague at IBM - "there is no secret silver bullet to really good compilers, it's just a lot of long hard work". If you keep resetting that hard work every couple years, that seems ... silly.
TL;DR If you really believe they've totally gotten everywhere they need to be in 3 months, i've got a bridge to sell you
[1] For example, good loop vectorization and SLP vectorization is hard.
https://webkit.org/blog-files/kraken.png
https://webkit.org/blog-files/octane.png
seriously though, dang, how many years of coding to get to that level of expertise
In addition to that, dedicated implementations can take various shortcuts to make their job easier - there are some nice examples given in the link. LuaJIT is example of a compiler project that benefits from being heavily specialised to a particular job, to remarkable effect.
