undefined | Better HN

0 pointskazinator4y ago0 comments

What ZFS will do it not capture a state before or after the transaction, but it will copy a state that the file actually has at some point during the transaction. The snapshot will have the state of the file which contains write_0, write_1, ...., write_k; and does not contain the effects of write_k+1, write_k+2, ...

The serial copy of the file does not have that property. We may be able to find some k such that the backup contains some writes newer than k, while missing some older ones.

The snapshot situation looks, to the application, like a crash and reboot, which it knows how to deal with (with test coverage and everything). The application recognizes that there is an uncommitted transaction and rolls things to the prior state. Or else sees that there is enough info to finish the transaction; just the commit part wasn't done.

The missing old writes situation doesn't seem easily recoverable; I have no idea how you would code defensively against that, other than assuming that the backup copy is made serially, and make sure that in every transaction, the writes occur in increasing offset order: the same direction. If the backup and transaction race in the same direction, then it's impossible for the backup to have "holes" whereby some older writes are missing.

0 comments

3 comments · 2 top-level

a13692099934y ago· 1 in thread

> assuming that the backup copy is made serially, and make sure that in every transaction, the writes occur in increasing offset order: the same direction. If the backup and transaction race in the same direction, then it's impossible for the backup to have "holes" whereby some older writes are missing.

Actually, that's the opposite of what you want: if the backup and application are racing in the same direction, then every write has a coinflip opportunity to either be included in the backup (if the application is slightly ahead and writes that block just before the backup reads it), or be ommited from the backup (if the backup is slightly ahead and reads that block just before the application writes it).

Whereas if the application writes from the end of the file to the start, the write where it meets the backup coming the opposite direction will be arbitrarily kept or lost, but in theory (but not in practice), any writes before it (toward the end of the file) will always be kept, and any writes after it (toward the start of the file) will always be lost, which is the same semantics you'd get from a snapshot at the time of that write.

kazinatorOP4y ago

You're right; racing in the same direction means there can be unwanted "holes": old blocks restored. E.g. database writes blocks 1 2 3 4; then the backup takes over and copies 1 2 3 4 5 6. Then database writes 5 6 7 8. Now 5 6 are new, and not backed up. Backup copies 7 8. Those are new and backed up. So 5 6 are a "stale hole" in the backup. We avoid this problem with opposite order; they can cross paths at most once.

yakubin4y ago

You're right. (Didn't notice your reply before updating my comment.)

j / k navigate · click thread line to collapse