undefined | Better HN

0 pointsloeg3y ago0 comments

My impression was that lz4 ratios were still marginally better than zstd for the same compression speed, and decompression is much, much faster.

0 comments

klodolph3y ago

We must be looking at different graphs. See p.7

https://indico.fnal.gov/event/16264/contributions/36466/atta...

You can see the classic Pareto frontier, with LZ4 filling the niche at the very bottom right edge of the graph.

loegOP3y ago

I'm looking at http://facebook.github.io/zstd/ "Benchmarks."

P.7 of your slides doesn't seem to cover Zstd --fast or decompression. It would be interesting to see how Zstd performs in those modes.

rincebrain3y ago

Depending on your data, experimentally, some people find zstd --fast can beat LZ4 for them on compression, some people find the opposite; my usual advice to people considering one or the other is to experiment and find out.

(An interesting anecdote about the differing notions of compressibility - when I recently wrote something to do a clever dance to avoid burning a great deal of CPU on incompressible data with higher zstd levels, I ended up using LZ4 -> zstd-1 as a two-tier filter to catch incompressible data, because what they each thought was incompressible was different enough that only using LZ4 lost a significant amount of compression sometimes, but only using zstd-1 was comparatively expensive and also lost a significant fraction.)

adgjlsfhk13y ago

note that for decompression speeds, they quote "We know LZ4 is significantly faster than ZSTD on standalone benchmarks: likely bottleneck is ROOT IO API"

j / k navigate · click thread line to collapse