undefined | Better HN

0 pointsDylan168071y ago0 comments

Because it seems very unfair to call it "filename shenanigans". The filenames are only there to point the script at the right file. Filename shenanigans would be something like putting actual bytes into the filename.

If you patched `cat` to ignore filename and just spit out each file as given, the script would still work without a single change. If you slightly changed the script to loop over the results of `ls`, it could still be compatible with scrambled filenames.

A script that didn't cheat would also be using filenames to a similar level.

In other words the filenames are a completely fair implementation detail. And that detail can be swapped out without changing the trick in any meaningful way.

The trick is based on having a series of variable-sized blobs of bytes. That's all it needs. If I use javascript instead of sh, and my decompressor is `[...s].join('5')`, I'm using the same trick.

0 comments

anamexis1y ago

> If you patched `cat` to ignore filename and just spit out each file as given, the script would still work without a single change. If you slightly changed the script to loop over the results of `ls`, it could still be compatible with scrambled filenames.

This isn't true! If you scrambled the filenames, the files would be put together in the wrong order and the result would be incorrect. You would need to also transmit the order that the files would be put together separately, which again, together with the size of the files themselves, would be greater than the size of the output.

The key thing here is that the trick works by storing the information of how the blobs are ordered out-of-band. In the OP, that out-of-band place to store the blob order is filename. In your JS example of `[...s].join('5')`, where does the order of [...s] come from? It's not something you can hand-wave away, it's the key thing that makes the trick work.

Dylan16807OP1y ago

> This isn't true! If you scrambled the filenames

I said "could" because you'd have to either do a limited scramble or hotwire ls to use the right order despite the scrambling. Or sort by date or inode, probably.

> The key thing here is that the trick works by storing the information of how the blobs are ordered out-of-band.

Yes. That is the key, not the filenames.

> In the OP, that out-of-band place to store the blob order is filename.

It is, but the actual use of filenames is not a shenanigan, and the blob order could be easily accomplished without any particular filenames.

> In your JS example of `[...s].join('5')`, where does the order of [...s] come from? It's not something you can hand-wave away, it's the key thing that makes the trick work.

It comes from the process of loading the blobs onto the computer. I'm not trying to hand-wave it, I'm saying it doesn't need filenames or anything resembling filenames. Maybe it came from a tar. Maybe I sent each file in a separate email. All that matters is having an order, and having an order happens by default when you have multiple files. As long as you don't go out of your way to reorder things, the trick works.

anamexis1y ago

> It comes from the process of loading the blobs onto the computer. I'm not trying to hand-wave it, I'm saying it doesn't need filenames. Maybe it came from a tar. Maybe I sent each file in a separate email. All that matters is having an order, and having an order happens by default when you have multiple files. As long as you don't go out of your way to reorder things, the trick works.

I guess that's where we disagree. I think you don't have an order by default, you need to explicitly define it, and transmit it, and store it somehow. Which is after all, why it's not true compression. When you account for that metadata, the "compressed" data is not smaller than the original.

In the OP, the cheat was using filenames to store that data. In a tar file, it's using the tar file metadata to store it. In your email, you're storing the email metadata to keep that ordering. In all cases, order is a key thing that you need to explicitly define, transmit, and store. And in all cases, this metadata takes up more space than is saved by the whole scheme.

1 more reply

j / k navigate · click thread line to collapse

0 pointsDylan168071y ago0 comments

A script that didn't cheat would also be using filenames to a similar level.

In other words the filenames are a completely fair implementation detail. And that detail can be swapped out without changing the trick in any meaningful way.

The trick is based on having a series of variable-sized blobs of bytes. That's all it needs. If I use javascript instead of sh, and my decompressor is `[...s].join('5')`, I'm using the same trick.

0 comments

anamexis1y ago

Dylan16807OP1y ago

> This isn't true! If you scrambled the filenames

I said "could" because you'd have to either do a limited scramble or hotwire ls to use the right order despite the scrambling. Or sort by date or inode, probably.

> The key thing here is that the trick works by storing the information of how the blobs are ordered out-of-band.

Yes. That is the key, not the filenames.

> In the OP, that out-of-band place to store the blob order is filename.

It is, but the actual use of filenames is not a shenanigan, and the blob order could be easily accomplished without any particular filenames.

> In your JS example of `[...s].join('5')`, where does the order of [...s] come from? It's not something you can hand-wave away, it's the key thing that makes the trick work.

anamexis1y ago

1 more reply

j / k navigate · click thread line to collapse