undefined | Better HN

0 pointsmpeg3mo ago0 comments

if one of the advantages is making it copy-pastable then I would suggest the REXC viewer should give you the option to copy the REXC output, currently I have no way of knowing this by looking at your github or demo viewer

another thing, I put in a 400KB json and the REXC is 250KB, cool, but ideally the viewer should also tell me the compressed sizes, because that same json is 65kb after zstd, no idea how well your REXC will compress

edit: I think I figured out you can right click "copy as REXC" on the top object in the viewer to get an output, and compressed it, same document as my json compressed to 110kb, so this is not great... 2x the size of json after compression.

0 pointsmpeg3mo ago0 comments

0 comments

4 comments · 2 top-level

creationix3mo ago· 2 in thread

> 2x the size of json after compression

Right and that makes sense. There is more information in here. The entire thing is length prefixed and even indexed for O(1) array lookups and O(log2 N) object lookups.

If you don't care about random access and you don't mind the overhead of decompression, don't use RX.

mpegOP3mo ago

I think this makes sense, when you explain it like that, it might be a matter of cleaning up the docs a bit so the "why" of RX is more clear (admittedly, a README is not always the best channel for this!)

creationix3mo ago

I've rewritten the framing in the README to first explain when you should use RX and when you should not. Most uses of JSON should probably stay JSON.

Let me know what you think

https://github.com/creationix/rx/blob/main/README.md#when-to...

creationix3mo ago

Thanks for testing it out! Yes, the website could use some love to make everything more discoverable.

The primary use case is not compression, it's just a nice side effect of the deduplication. This will never beat something like zstd, brotli, or even gzip.

My production use cases are unique in that I can't afford the CPU to decompress to JSON and then parse to native objects. But with this format, I can use the text as-is with zero preprocessing and as a bonus my datasets are 18x smaller.

j / k navigate · click thread line to collapse