undefined | Better HN

0 pointshuhtenberg12y ago0 comments

Of course it CAN and it DOES NOT. If I flip two bits in a large file - one at the head and at the tail - then no matter how clever the algorithm is, the update cannot be atomic without proper support from the OS, because it would involve two separates Writes into the file.

On Windows there's Transactional NTFS whereby you can bind an open file to a transaction and then have either all or no changes applied at once. But that's only Vista+ and I am pretty sure rsync doesn't use it anyhow.

0 comments

beagle312y ago

Flip those two bits. What rsync will do on the target system is create a copy of the file you want (with a name like .tempxasdiohkshlksdf-filename.ext) which takes most of the data from the local copy, and a few kilobytes of patches transferred. Then, when this file has been created, closed, its attributes properly set, and it is an identical copy of the file on the source system - it will rename ("move") the temporary file into the name that it should have. This move operation is what makes everything atomic;

It does cost another copy of the file on disk, but it does NOT leave the file in an inconsistent state. It is either the original file, or the new file - no in between.

You CAN avoid this behavior, by using the "--inplace" switch or the "--append" switch, which tell rsync to just modify the file in-place. However, this is NOT atomic, and NOT the default (for that exact reason).

huhtenbergOP12y ago

OK, you win. I didn't realize rsync was solving for network-bound scenarios, but in retrospect it makes sense.

j / k navigate · click thread line to collapse