Apple treats tar less like “portable Unix interchange” and more like “archive this filesystem object faithfully.” That is very Apple, and very libarchive. ;-)
This is probably going to get worse (as Apple continues to add macOS-specific metadata), so your workaround is very helpful.
I haven't tested it in a while, but at one point, setting the COPYFILE_DISABLE=1 env variable would disable the inclusion of macOS-specific metadata.
If I point "tape archive" at a file system, I want that file system archived to tape. And so, tar does.
If I don't, well, that's a fine option, and there's a fine option for that.
So it's less of a "workaround" or something that "gets worse", than, "No, I don't really want a tape archive of this filesystem, only of some of it." And that's supported.
That said, never seeing another .DS_Store should be a system-wide option!
Principle of least surprise is good engineering practice. The question is always whose surprise. Someone who expects tar to behave like other UNIX systems is going to be surprised by this. Someone who expects tar on Apple to have perfect fidelity would be surprised by not-this.
I increasingly feel like build systems should never be relying on any "native" utilities from the host system, and should instead be bringing them in via dependencies. You can't have this problem if your packaging system pulls in a specific portable `tar` library.
The traditional UNIX tar and cpio utilities cannot archive the modern Linux file systems without loss of metadata.
Most modern tar programs implement various file format extensions as a workaround for this, but the extensions may be incompatible between distinct tar programs and frequently they are very poorly documented.
Some years in the past, libarchive was the only archiver available on Linux that guaranteed lossless backups for the Linux file systems, e.g. xfs or ext4 (and also lossless file transfers between Linux file systems and FreeBSD file systems). Therefore that is what I have been using on Linux since then.
Presumably since then GNU tar and other tar programs should have caught up with it, but I have not verified this.
Whichever tar program was used in TFA, it was an obsolete tar program, and that was the real problem, not that the archives had been created on an Apple computer.
Yes please.
Linux developers already do. Using a BSD can already be a pain in the arse, thanks to (often poorly thought out) Linux-isms cropping up everywhere.
The problem described in TFA is not specific to Apple, but the same problem appears when archiving any decent filesystem that has been designed during the last 3 decades and not a half of century ago, including all Linux file systems.
The problem described in TFA is not caused by Apple, but by the author using an obsolete tar program and not being aware of this.
The traditional tar file format cannot store a lot of the metadata that is contained in modern file systems (e.g. high resolution timestamps, access control lists, extended file attributes), so it is useless for such file systems.
Most modern "tar" implementations have added extensions to the tar file format, to make it usable with modern file systems, such as Linux XFS or Linux EXT4. But many of these extensions are incompatible between themselves, so certain tar files can be fully extracted only with the same tar program that has created them.
I strongly recommend against using the old tar or cpio file formats. Even with various extensions it is not guaranteed that they always work correctly.
I always use only the pax file format, which has also required extensions in order to work with the modern file systems, but the pax extensions are cleaner than those for tar, because the file format is better designed.
Libarchive, which was mentioned in TFA, is available in most Linux distributions or it can be built from source on any Linux computer. It provides an executable that is preferable to tar (better invoked as "bsdtar --format=pax") for the backup or transfer of any Linux files.
I have not checked recently GNU tar or other tar programs available on Linux, and I hope that meanwhile they have been upgraded to be able to archive losslessly the Linux file systems, but some years ago that was not true, so using tar or cpio on Linux could easily corrupt the archived files.
The warning can be suppressed by `--no-xattrs --no-mac-metadata`.
- tar czf dist.tar.gz dist
+ COPYFILE_DISABLE=1 tar czf dist.tar.gz distMy computer should run programs when I tell it to run them.
Don’t blunt _every_ tool just to make them harder to cut yourself on.
Provenance information bundled into a file is not the worst idea in the world IMO. We have created/modified timestamps on files already, right? There's definitely the question of "why" but hey if more of my binaries just had at least a tag about who put them there that would be a win in my book.
Not an argument for doing what MacOS does, just an argument that the info would be nice to have.
Open question, is it worth attempting to main these semantics between mac and linux.
Linux users generally install software with apt or rpm. Or steam.
The existence of any executable file outside the system dirs it a red flag in itself.
tar --no-xattrs --no-mac-metadata -czf $ which gtar
gtar is /opt/homebrew/bin/gtarThese are not errors. They are simply warnings about extended attributes being ignored when extracting files, which seems completely fine to me, and creating the tar without those extended attributes has exactly the same outcome, but throws away the metadata at archive time instead of extraction time.
Furthermore, this is not an Apple/macOS issue. The tool used is bsdtar, so it would also affect all BSD-variants that default to bsdtar/libarchive, and those systems also have extended attributes, e.g., for SELinux, which would get added to the TAR.
* https://mgorny.pl/articles/portability-of-tar-features.html#...
* Via: https://github.com/mxmlnkn/ratarmount/issues/145
bsdtar ≥3.7.2 apparently adds both types to its files for maximum portability:
* https://github.com/libarchive/libarchive/pull/691/files#diff...
AFAICT, bsdtar will default to "ustar" format, but will auto-switch to "pax" if needed.
> For some reason
Very informative!
One thing that still trips me up though: `._Foo.txt` AppleDouble files get created in your filesystem any time something Finder-adjacent touches a folder, and gtar archives them just fine, but they show up as garbage on the Linux side. `dot_clean -m mydir/` before tarring kills them, or you can pipe through `--exclude='._*'` if you don't want to touch the source tree.
But still interesting nonetheless why they are added
Don't forget the MacBook Neo's A18 Pro :)
What horrible advice also to download different tar versions, for something that should just be explained properly.
If it weren't for the "2024" in the title, I would have thought this to be a result from AI.
But it's not artificial intelligence. It's real stupidity.
Oh...