Unison was the first backup binary that we built into the rsync.net platform - breaking our original design goal of only offering client agnostic SSH and the tools that would run over that.
Shortly afterward we also added rdiff-backup. Both of these tools were quite popular and we saw a lot of interest back in 2005 - 2010 but we see very little use or interest in them now.
All of the interest in backup clients is now in rclone[1], restic[2] and borg[3].
restic was easy - you can point it at any SFTP capable host.
borg and rclone, on the other hand, we had to (like unison and rdiff-backup) build and maintain on the rsync.net server side.
All of these (save rclone, which is a binary executable) are python scripts. But we don't have a python interpreter (or any interpreter) in our very locked down platform. Can anyone guess how we do that ?
[1] https://github.com/rclone/rclone/issues/3254
Like you did with attic and borg? Quoting you on January 2016:
> We solved the problem by (cx)freezing the attic and borg python tools into binary executables. So, still no python in our environment (reducing attack surface) but the ability to run attic and borg just like they are meant to be run.
As seen here: https://news.ycombinator.com/item?id=10925123
But Unison is for syncing, which means maintaining eventually consistent replicas of a changing directory tree (up to some pragmatic exceptions and manual tweaks). The whole point of Unison is to turn multiple devices into a single failure domain, which requires a separate system for storing safe backups.
Also, open source but non free license but duplicacy, in my opinion is technically superior to restic.
In Fedora (and I think this applies in Debian too) we have to maintain 3 versions because Unison isn't interoperable across minor releases. For this reason we package 2.13, 2.27 and 2.40, and I think there is discussion about packaging the latest release too. Keeping these ancient (esp 2.13) versions going is a pain to say the least.
The last straw came when it turns out that unison was incompatible with the same version, if that version had been built on a different system. I can't remember the details but there was some library version difference that results in a unison that has the same version number but wouldn't talk to one from another system without crashing.
Drove me crazy.
My use-case is that I run Unison on two computers running Fedora, to implement a form of "sneakernet" file sync. Work computer <-> usb thumb drive <-> home computer.
I wasn't aware that the exact version of Unison was so important. Will start paying attention to that.
Ouch! FWIW, Thanks for all your work.
On Fedora I'm using unison251-text from the croadfeldt/Unison COPR. On Gentoo it's net-misc/unison-2.51.2, on FreeBSD it's unison-nox11-2.51.2 from pkg.
But you package for Fedora, which is "upstream" for RHEL (if I'm not mistaken) with their 10 year life cycles, so I'm sure RHEL customers would love you to keep as many versions available as possible :)
I use Syncthing now. It's really useful.
(That said, one of the problems I encountered with Syncthing seemed to be that it wouldn’t properly reconnect when I switched networks, which is necessary since it runs as a persistent daemon. With Unison, so far I’ve been starting it up in the foreground in a terminal as needed, so it hasn’t had to provide the same functionality. But I expect I’ll eventually set up some type of auto-reconnect, which will hopefully work better than Syncthing did.)
When go to another country, open my laptop, connect to a wireless network and bam. Syncthing connects, things got synced automagically.
I reluctantly installed Dropbox one day after forgetting my external drive at home. Honestly, that was the beginning of a miserable time. Dropbox never really delivered, it took forever to sync, and constantly caused the internal disk to trash. I had tons of other problems with Dropbox over the years and was very happy the day I quit using it.
I still have a unison alias in my .profile
alias unison="unison -logfile /Users/eddie/log/unison.log"
- Unison is written in OCaml, which is (probably) a perfectly fine language but not commonly used
- Unison synchronizes files to files, but for a Dropbox-like system you really want deduplication for space savings (i.e. server-side storage is a bunch of pointers to content-addressed blocks.)
- in general, it's not clear to me that the client is really the hard part of Dropbox. Note e.g. the part where Dropbox now runs its own data centers for cost reasons.
- there's a million fiddly things to get right, and Unison hasn't had that much usage
Unison is the only bidirectional sync tool that I trust to get the details right. It is backed by a formal model with various proofs of correctness. Such models are also easily representated in OCaml; which I can assure you is more than a fine language, especially if one cares about correctness. Dropbox has struggled to get these details correct in the past (see the paper "Mysteries of DropBox", by Unison's author Professor Benjamin Pierce). TLDR, Pierce teaches these Python hackers how to fix their broken code.
The whole point of Unison is that you do not need a server. And certainly not a server run by a for-profit corporation in the United States of Surveillance.
I'd be really happy to receive feedback if you have a chance to try it out!
You can also orchestrate your setup a bit (e.g. to work with Docker Compose) using Mutagen's new orchestration infrastructure. You can find an example of this at the bottom of the Mutagen homepage: https://mutagen.io
> Further improvements to the OS X GUI (thanks to Alan Schmitt and Craig Federighi).
Here are Craig's commits:
- https://github.com/bcpierce00/unison/commit/48f8e1b27edbe2df...
- https://github.com/bcpierce00/unison/commit/6645d1793ce843f6...
Around the same time, Dave Abrahams, now of Apple's Swift team, makes an appearance on the unison-hackers mailing list.
There is one issue that I have run into though. If I format the USB as fat32 it can't properly store permissions. And if I format it as ext4 then I need to make sure that my numeric user ID is exactly the same in both my home and work computers (otherwise the permissions get messed up).
Is there a way to make my arrangement more robust so that I can sync files through and USB drive, without needing to use the same numeric user id in all my computers?
$ chown -Rf ufo.users /media/usbdrive && unison
The problem with trying to edit these files was that everyone logged in on the network had access to them and could change them arbitrarily. Or they could even just inadvertently lock the files if they left them open on their PCs. There was no control or change management.
I decided to keep the 'canonical' versions of the files on my PC at work and use Unison to sync them over to the shared network versions of the files once a day. Unison would instantly tell me if anyone other than me had changed the files, and I could investigate further. It was a huge relief knowing that I had proper control over those files.
[0] https://wiki.archlinux.org/index.php/Unison#Version_incompat... , https://groups.yahoo.com/neo/groups/unison-users/conversatio...
Something I love about Unison is the fact that I can sync files from anywhere on my filesystem without having to move them or replace them with symlinks.
I can also sync subsets of my files by defining multiple profiles. I have Unison set up to default to a "common" profile that includes nearly everything, but I can explicitly request a sub-profile if I want. I also have a separate profile that syncs the profiles themselves, so I can make profile changes on one machine and run `unison profiles` to upload them.
I often evaluate a wide range of software before choosing what I consider best for a job, and many years ago, Unison came out way ahead in such an evaluation. It never failed me.
A hint regarding running the latest Mac binary on Mojave: if the GUI crashes on you, it's likely to be an issue regarding syncing file _permissions_ and not just files.
This has been biting me for a couple of months now whenever I sync my development tree between Dropbox and OneDrive, since Windows' WSL tends to set wonky 0x777 permissions on entire file trees, and it is usually those that cause Unison to crash.
Otherwise, I've been using it to sync 400K+ files without incident.
A long time ago used unison to keep file uploads on 2x primary web servers[1] in sync. It worked like a charm. With IIS on Windows!