A Real-World WebAssembly Benchmark (opens in new tab)

A big part of the score is compilation time, which improves a lot with Liftoff, our baseline compiler for WebAssembly in V8. We've been in contact with the benchmark authors and have identified other bottlenecks in our engine (e.g. instantiation time), which we are now working on optimizing. Look forward to some increasing scores soon :)

Pretty soon we will be running Chrome on Firefox.

IIRC they've also always had the fastest Typed Arrays. I wonder if there is a connection there - it wouldn't surprise me if memory access is the biggest bottleneck for WASM in other browsers.

Have you tried enabling the experimental new WebAssembly baseline compiler in Chrome Canary?

On my phone (OP6) FireFox beats Chrome by two whole seconds

Samsung Galaxy S9+ (w/ Exynos):

  Firefox: 6900
  Firefox Nightly: 7874
  Google Chrome: 10602

Ha! Not sure if you've seen it, but we offer native SDKs for iOS, Android, Windows, macOS and Linux. Our Web SDK also has a version that uses PSPDFKit Server via a Docker container to get faster performance.

The performance difference to native is smaller than you might think though - see our post from a few months back: https://pspdfkit.com/blog/2017/webassembly-a-new-hope/

PSPDFKit for Web Standalone is a great choice for an easy-to-deploy web viewer, annotation editor and form filler, and performance is great. Definitely better than pdf.js - which is typically where people start with, before they upgrade to PSPDFKit for Web.

On my computer, the native PDF reader is xpdf, which is made of security holes, or something based on libpoppler, which is made of patches to xpdf.

Right? The built in safari reader does everything correctly and interacts with the rest of the OS properly. I assume that the built in chrome one also Just Works (tm).

Also why would I want to download an entire pdf engine when i already have an AOT version? (Stealing that phrasing by the way :) )

I prefer the sandboxed PDF reader in my browser to the native one that's not sandboxed

https://www.theregister.co.uk/2016/10/24/apple_security_upda...

Hello Martin - slightly off topic. There's no pricing section on your website. Does one have to get in touch with sales for a quote?

Hey, thanks for the interesting article, and the benchmark of course!

One bit of constructive feedback: add a date of publishing to the article. PSPDFKit already has multiple articles on WASM before, and it's nice to be able to tell which one is the most "up to date" at a glance.

Any chance you also benchmarked WA version against native code? What are results in your case?

Hi there,

Thanks for publishing this benchmark, but I've got a question about the results. The description of the charts states that 'A lower score is better', and between the two charts you say 'when we compare WebAssembly with our JavaScript fallback ... all browsers except Firefox are still slower'. But WebAssembly has a lower score than JavaScript on all browser/OS combinations except Chrome67/macOS, Safari/macOS and Edge/Win10. Could you clarify which way the results are pointing?

Can you post the code somewhere? I'm very interested in this.

and whats the performance using native code(c/c++)? Im guessing ~0.1sec on same models.

For geometry heavy stuff, web workers (possibly in addition to wasm) be super helpful in preventing the browser from blocking while number crunching.

In my understanding, SIMD is an example of a feature like this, as is threading. It’s not just Microsoft that wants this.

>This is very vague, but already disturbing

Yeah, god forbid the platforms gets more powerful...

Maybe they'll be the first to implement tail calls!

I'm guessing these "features" will be tied to the Microsoft Store, something they're already doing with PWAs.

Many were quick to criticize Valve and Epic about their own criticism of the Microsoft Store, which is why Microsoft is now using a "boiling the frog" strategy to tie everything to the Store over the next decade or so. And this process will likely accelerate once Windows 7 is dead and buried.

Since all languages should be able to compile to wasm, that question is pretty close to "Which language is the most performant?", which is a can of worms that I'm not going to open. Maybe something like the Benchmarks Game would be useful here. Or already exists, and someone could point me to it :)

A similar question would be "What languages do you know of that are focused on performance and compiling to wasm?" For that, I'd say C++, Rust, and Go, though I'm sure there are others.

It's really hard to say, especially as the wasm interpreter stuff itself is under so much flux; different implementations act differently, etc.

There's also the question of what aspect of performance is of interest to you. Do you include the download time in performance? If so, languages that produce leaner wasm files will be better performing, even if the execution is identical. Do you only care about execution time? Then that aspect isn't relevant. etc.

That depends on (a) how you define your measure for "performance" (see other comments that talk about the difference between microbenchmarking and application benchmarking), (b) the compiler that you use, (c) the WebAssembly implementation that the code finally runs on.

So, do not expect a clear answer to that questions. Probably no one really knows because of a lack of benchmarks and deployments in the wild that have been analyzed. That's why they made the benchmark.

If you really do mean fastest and nothing else, you can't really beat WAT, the WebAssembly text format.

Are you sure gl.commit is WebAssembly only? That doesn’t seem right to me, but I could be wrong: https://hacks.mozilla.org/2016/01/webgl-off-the-main-thread/

What is TPTB?

SIMD is another WebAssembly only API, hopefully coming soon.

One is a large c++ compiled ti asm.js and wasm. The other is wasm compiled rust vs javascript. How are they comparable?

If you render to canvas, you would need to reimplement scrolling and zooming, checkboxes and input elements. Any custom reimplementation of this stuff will end up slower and clunkier than the optimised, native versions already in browsers.

Also (as far as I know) canvas content is inaccessible to screen readers. Accessibility matters.

Metrics are required platforms (iOS, Android, Web, Windows, macOS, Linux), components (see matrix on https://pspdfkit.com/pdf-sdk/), number of users, number of server nodes (if Docker-based), number of apps/[sub]domains, internal/external access and expected support, to name a few. We're usually quite fast to figure something out that works for both parties, pricing is in the 4 (passion projects) to 6 (enterprise) figure range. Simply drop us a message via https://pspdfkit.com/sales to get things started. We try to make this process as easy as possible.

You already get 2D, WebGL, WebGL 2 through canvas

Looks like they are working on it, which is good news ->

From https://twitter.com/steipete/status/1013035117076467713 :

Shoutout to the Safari team! They were able to track down the slowness in the current WebAssembly implementation and ALREADY FIXED IT. Patch is here: https://trac.webkit.org/changeset/233378/webkit

(trace points caused a significant (4x!) slowdown as our benchmark performs many calls into the VM)

In case anyone else was wondering, you actually CAN do this and JIT WASM in your browser via Binaryen:

https://github.com/WebAssembly/binaryen/blob/master/test/bin...