WebP: The WebPage Compression Format (opens in new tab)

There is an interesting "Save-Data" header to let a site know which makes sense to optimize for on connection but it seems to be Chrome only so far https://caniuse.com/?search=save-data

I wish there was a bit of an opposite option - a "don't lazy/partially load anything" for those of us on fiber watching images pop up as we scroll past them in the page that's been open for a minute.

> ... on roaming data.

A little OT, and I'm not sure if iOS has this ability, but I found that while I'm traveling, if I enable Data Saver on my (Android) phone, I can easily go a couple weeks using under 500MB of cellular data. (I also hop onto public wifi whenever it's available, so being in a place with lots of that is helpful.)

My partner, who has an iPhone, and couldn't find an option like that (maybe it exists; I don't think she tried very hard to find it), blew through her 5GB of free high-speed roaming data (T-Mobile; after that you get 256kbps, essentially unusable) in 5 or 6 days on that same trip.

It turns out there's so much crap going on the background, and it's all so unnecessary for the general user experience. And I bet it saves battery too. Anything that uses Google's push notifications system still works fine and gets timely notifications, as IIRC that connection is exempt from the data-saving feature.

I've thought about leaving Data Saver on all the time, even when on my home cellular network. Should probably try it and see how it goes.

But overall, yes, it would be great if website designers didn't design as if everyone is on an unmetered gigabit link with 5ms latency...

We need a MoSh-based browser with gopher support.

I do wish site authors wouldn't waste my bandwidth, but I wish much more they wouldn't waste my CPU and break usability in various ways like TFA's "solution" does. This is not a win even if you care deeply about optimizing sites.

Other major issues that I had to contend with:

1: browsers choose when to download files and run JavaScript. It is not as easy as one might think to force JavaScript to run immediately as high priority (which it needs to be when it is on critical path to painting).

2: you lose certain browser optimisations where normally many things are done in parallel. Instead you are introducing delays into critical path and those delays might not be worth the "gain".

3: Browsers do great things to start requesting files in parallel as files are detected with HTML/CSS. Removing that feature can be a poor tradeoff.

There are a few other unobvious downsides. I would never deploy anything like that to a production site without serious engineering effort to measure the costs and benefits.

Now I got curious, and there's also a 400 kB CSS file to go with it: https://purplesyringa.moe/fonts/webfont.css

I'm not up to date on web/font development – does anybody know what that does?

seriously, why can't modern browsers turn off features like remote fonts, webrtc, etc. in settings. I hate when reading a bit then the font changes. Not to say fingerprinting risks.

Even the gzipped page itself could be shrunk by 40-50% by optimizing the inline SVGs.

Just removing the inline duplicated SVGs used for light/dark mode and using styles instead would bring it down 30%. Replacing the paths used to write text (!) with <text> and making most of the overly complicated paths into simple rect patterns would take care of the rest.

Wow, yeah. That kind of discredits the blog author a bit.

You don't need a new roundtrip for every packet. That would be devastating for throughput. One vs two vs three file packets get acked as a batch either way, not serially.

Also when you get to the end, you then see

> The actual savings here are moderate: the original is 88 KiB with gzip, and the WebP one is 83 KiB with gzip. In contrast, Brotli would provide 69 KiB.

At 69 KiB you're still over the default TCP packet max, which means both cases transmit the same number of packets, one just has a bunch of extra overhead added for the extra JavaScript fetch, load, and execute.

The time saved here is going to be negligible at best anyway, but there looks to be actually negative because we're burning time without reducing the number of needed packets at all.

Interesting, how does that add a round trip? For the record here's what I believe to be the common definition of an additional "round trip", in a web development context:

  - client requests X
  - client gets X, which contains a reference to Y
  - therefore client requests Y

So you're starting a new request that depends on the client having received the first one. (although upon closer inspection I think the technique described in the blog post manages to fit everything into the first response, so I'm not sure how relevant this is)

If the blob contains requests (images, but also stylesheets, JS, or worst case fonts), it will actually instead be a net negative to latency. The browser's preload scanner begins fetching resources even before the HTML is finished being parsed. That can't happen if the HTML doesn't exist until after JS decodes it. In other words, the entire body has become a blocking resource.

These are similar conversations people have around hydration, by the by.

It takes more time for your message to get back and forth between your computer and the server than it takes for the server to pump out some extra bits.

Even if you transmit the js stuff inline, the op's notion of time still just ignores the fact that it takes the caller time to even ask the server for the data in the first place, and at such small sizes that time swallows the time to transmit from the user's perspective.

Hmm? I'm not sure where you're taking that from. The webp is inlined.

XZ was never considered to become a compression algorithm built into web browsers to start with. Brotli decoder is already there for HTTP, so it has been proposed to include the full Brotli decoder and decoder API as it shouldn't take too much effort to add an encoder and expose them.

Also, XZ (or LZMA/LZMA2 in general) produces a smaller compressed data than Brotli with lots of free time, but is much slower than Brotli when targetting the same compression ratio. This is because LZMA/LZMA2 uses an adaptive range coder and multiple code distribution contexts, both highly contribute to the slowness when higher compression ratios are requested. Brotli only has the latter and its coding is just a bitwise Huffman coder.

I can shrink any file down to just 32 bits using my unique method.

I call it the High Amplitude Shrinkage Heuristic, or H.A.S.H.

It is also reversible, but only safely to the last encoded file due to quantum hyperspace entanglement of ionic bonds. H.A.S.H.ing a different file will disrupt them preventing recovery of the original data.

That's an interesting concept, although it would generate a ton of bogomips since each client has to generate the image themselves instead of one time on the server.

You'd also want "seed" and "engine" attributes to ensure all visitors see the same result.

That would require a lot of GBs in libraries in the browser and a lot of processing power on the client CPU to render an image that is so unimportant that it doesn't really matter if it shows exactly what the author intended. To summarize that in three words: a useless technology.

That idea is something that is only cool in theory.

Similar to what Samsung does if you take a picture of the moon.

Similar ideas have floated around for a while. I’ve always enjoyed the elegance of compressing things down to a start and end index of digits of Pi.

It’s utterly impractical, but fun to muse about how neat it would be if it weren’t.

This messages comes to you via a 10Gbps, $50 residential fiber connection.

The future is already here – it's just not very evenly distributed.

Very relatable! "If this MD5 hash uniquely identifies that entire movie, why would anyone need to ever send the actual... Oh, I get it."

Arguably the cruelest implication of the pigeonhole principle.

Even better, take a physical object and slice it precisely in a ratio that contains your data in the fraction!

Brotli and zstd are close, each trading in various configurations and cases - but zstd is muuuuuch more usable in a practical sense, because the code base can be easily compiled and linked against in a wider number of places with less effort, and the cli tools turn up all over for similar reasons. Brotli like many Google libraries in the last decade are infected with googlisms. Brotli is less bad, in a way, for example the C comes with zero build system at all, which is marginally better than a bazel disaster of generated stuff, but it’s still not in a state the average distro contributor gives a care to go turn into a library, prepare pkgconf rules for and whatnot - plus no versioning and so on. Oh and the tests are currently failing.

Realistically, everything should just support everything. There’s no reason why every (full featured) web server and every (full featured) web browser couldn’t support all compelling data compression algorithms.

Unfortunately we live in a world where Google decides to rip JPEG-XL support out of Chrome for seemingly no reason other than spite. If the reason was a lack of maturity in the underlying library, fine, but that wasn’t the reason they offered.

Why would you do this?

A different comment says librewolf disables webgl by default, breaking OP's decompression. Is that what you're seeing?

Works fine on Safari and Firefox for me.

WebP is already deprecated from a compression ratio standpoint - it's entirely surpassed by all of AVIF, HEIF, and JXL. Too bad that none of these has broad support (and the best of them, JXL, is being blocked by all browser makers).

WebM on the other hand still has a reason to exist unfortunately: patents on H.264.

Conversion means additional quality loss unless you are converting to a lossy format. Better solution would be to use reasonable tools that are not stuck in the stone age and can open WebP directly.

I was wondering that too, but that dictionary itself compresses down to 29 KB (using regular old gzip), so it seems pretty lightweight to include even if it were hard/annoying to precompute at runtime or install time.

I'm actually not convinced that there are two different dictionaries at all. The Brotli RFC only talks about a static dictionary, not a separate encoding vs. a decoding dictionary.

My suspicion is that this is a confusion of the (runtime) sliding window, which limits maximum required memory on the decoder's side to 16 MB, with the actual shared static dictionary (which needs to be present in the decoder only, as far as I can tell; the encoder can use it, and if it does, it would be the same one the decoder has as well).

It isn’t a serious proposal. It’s a creative hack that no one, author included is suggesting should be used in production.

This philosophy hands your content on a silver platter to ai companies, so they can rake in money while giving nothing back to the author.

Nothing technically objective, just the size that a typical photo can be reduced to without looking bad.

It really depends on what you're after, right? If preserving every detail matters to you, lossless is what you want. That's not going to create a good web experience for most users, though.

Yeah that's plausible, you definitely don't want any kind of untrusted data in the input.

Something I wanted to do but clearly never got around to, was figuring out how to put an open-comment sequence (<!--) in a header somewhere, so that most of the garbage gets commented out

This has stopped working many years ago. Every top-level domain now has its own private cache of all other domains.

You likely have dozens of copies of Google Fonts, each in a separate silo, with absolutely zero reuse between websites.

This is because a global cache use to work like a cookie, and has been used for tracking.

No, because of cache partitioning, which has been a thing for a while.

https://developer.chrome.com/blog/http-cache-partitioning

> webp has a lib on Linux but somehow, standard image viewers just do not read it

That may apply to old "LTS" Linuxes, but not any relatively recent one. Xviewer and gimp immediately come to mind as supporting it and i haven't had a graphics viewer on Linux _not_ be able to view webp in at least 3 or 4 years.

Huh, it seems it's just my browser that resets the noise every run.

I opened the page in Firefox like the article suggests and I get a different pattern per site and session. That prevents using the noise as a supercookie, I think, if its pattern changes every time cookies are deleted.

how does it affect error correction though?

Thanks