Apple File System Reference [pdf] (opens in new tab)

I was working on a video recorder product that could record videos directly to iPods, iPhones and other media players. iPods were formatted to HFS+, if you connected them to a Mac out-of-box, and were formatted to FAT32, if you connected them to a PC. The platform i was working on was an RTOS, and we had to develop both FAT32, and HFS+ from scratch. It was not read-only, it supported both read/write.

This is user space – hopefully a kernel level one will come out as a result of this.

Or does it matter, performance-wise? People that know more can chime in.

From the second paragraph of the PDF:

"This document is for developers of software that interacts with the file system directly, without using any frameworks or the operating system—for example, a disk recovery utility or an implementation of Apple File System on another platform."

On that note, I am surprised that they added Fusion-awareness to APFS, rather than just putting APFS on top of CoreStorage.

It certainly is better to have the filesystem aware of the Fusion situation, but...measurably, significantly better? Would the experience have been significantly worse without it? 10.13 betas allowed APFS use on Fusion drives, presumably without any Fusion-awareness in the FS.

I'm surprised, but happy to see they did it.

I don't defragment SSDs at all. I should've made that clear in my comment.

I have a multi-HDD system running to 6TB of storage, I can't run to the £ of an all-SSD system at the moment, but that's on the cards for the future, so it's spinning rust for me, for now.

Probably often but there is no way to tell really because the controller has a mind of its own and they are all different.

Here's the essential business logic specification for HFS+ automatic de-fragmentation, for the curious.

It's actually quite a clever spec, because it takes advantage of existing efforts to read fragmented files to perform the majority of the de-fragmentation process.

I'm not sure if this spec applies to APFS or for SSDs. (With SSDs you're generally better off not defragmenting most of the time, because the performance penalty is far lower, but the write amplification has consequences.)

——————

When a file is opened on an HFS+ volume, the following conditions are tested:

  If the file is less than 20 MB in size
  If the file is not already busy
  If the file is not read-only
  If the file has more than eight extents
  If the system has been up for at least 3 mins

If all of the above conditions are satisfied, the file is relocated—it is defragmented on-the-fly.

http://osxbook.com/software/hfsdebug/fragmentation.html

——————

A moderate amount of my work involves video training and the component parts to build, so raw video and audio. It's subjective, but there are times that video and audio stutters a bit and it's not down to the available CPU or RAM (plenty of headroom there).

My computer (macOS Sierra, HFS (Journaled)) has 2x spinning drives of 3TB, and operations are noticeably quicker after an overnight defragmentation run, which I do twice a year or so. Anecdata, granted - but it works for my situation.

You have not used a file system exceeding 90% use with large files and lots of little files coming and going.

ZFS in particular completely falls off a cliff somewhere between 80% and 90%, due to the copy on write nature of ZFS always allocating and freeing small bits of space. That creates the little gaps all over the FS which murder performance when the big gaps run out.

Simple, to help prevent “bit rot”. The problem is exacerbated further in that many of us treat cloud sync services as backup, which they arguably aren’t - they can inconveniently just spread the decay.

I’d also hoped that a next generation file system from Apple would have had more to say on this topic, but it seems like features that promote their iOS device agenda took front seat over less “sexy” features like data integrity.

In the days before iOS devices dominated OS level decision making at Apple there was an assumption that Apple might adopt ZFS as their next generation file system, which is apparently much better in this regard. There’s various evidence of a cancelled MacOS ZFS project scattered throughout past MacOS releases.

> https://en.wikipedia.org/wiki/Data_degradation

> https://arstechnica.com/gadgets/2016/06/zfs-the-other-new-ap...

Not OP, but probably to make sure the contents of a file are not changed by hardware errors.

I've recently listened through an old-but-good episode of the Hypercritical podcast with John Siracusa's informative rant about this very topic: http://5by5.tv/hypercritical/56

The pre-boot environment needs to find the kernel and initramfs somehow. My guess for how Apple is booting from APFS, now that it's all APFS, without a separate recovery partition? They've got this minimalist EFI jumpstart code in the firmware, it loads the EFI file system driver for APFS, and now it can locate the bootloader, kernel, and kext cache.

For a long time Apple has had an HFS+ driver baked into the firmware. The way APFS is implemented with EFI jumpstart, they've got much less filesystem code in firmware.

Every time I see the San Francisco font mentioned anywhere, I don't think of Apple's rather lovely neo-grotesque, but rather Apple's original San Francisco. (Which is also lovely, in an oh cool, this 1984 era computer has a variety of fonts kind of way.)

http://luc.devroye.org/SusanKare--SanFrancisco-1984.png

And San Francisco Mono for the monospaced font.