My dream setup would consist of a couple of low-fi workstations, say a bunch of Intel NUCs or the like, one at each site. Then, whenever I leave in the middle of work at one site, I can go to the other and pick up from there. However, due to sub-optimal internet connectivity I can’t use thin clients with a cloud-based X server. At each work location, I’d need to run my jobs on that machine (either bare metal, or virtualised). Is there a way to e.g. synchronise one virtual machine image across the workstations, when I switch locations? I wouldn’t need to run two workstations at once. Bonus points if a program or configuration, that I install one machine also shows up on the other workstations. I have no special requirements, any Unix-like OS with a window system is OK.
Is anyone of you running such a setup? Or have you found a similar solution? I have that nagging feeling I can’t see the forest for the trees, given that I haven’t with IT setups since 1999. Thanks!
Pure Git is great for consolidating configuration as well as keeping plaintext format files in sync. Invaluable if you are an org-mode or markdown jockey.
I've always wished there was something to help me sync my environments, especially between mac an Linux platforms (just sync my terms, keep the history, stuff like this), but at the end of the day, it forces me to stay clean and close my windows from time to time ...
Oh and I use Chrome with sync feature, and it works awesomely great! - with extensions like LastPass to sync all my accounts. I used to value privacy a lot .. I gave up when I moved to China.
- You could buy a more portable laptop that you wouldn't mind carrying around, for example the new Macbook Retina or an MS Surface.
- Even a poor Internet connection might still be good enough for command-line work, and you can get a lot done on the command line. I've had great success keeping all my code and data on a central server, and editing code either directly in command-line Emacs via mosh (https://mosh.mit.edu) or in a local Emacs instance using tramp (http://www.gnu.org/software/tramp/) to save changes directly to the server. This has the advantage that I can start long-running jobs that persist on the server even as I'm moving between locations. For data science work, iPython notebook is also very easy to set up for remote web access.
I work as much as I can on the command line but I have to use some visualisation tools. They are part of the problem: they take time to set up, require constant configuration tweaking and I see myself wasting hours keeping them in sync on my workstations.
Here I think you're setting yourself up for failure. What happens when said laptop is stolen, has a hard drive die, or you forget your encryption key? If it's more convenient for him to have multiple workstations, then why not also make sure he maintains a good practice of having his working data synchronized and therefore backed up?
For that reason, I probably would go with only syncing home/data dirs, and maybe doing so explicitly or using something like git-annex. If you want to sync installed packages and system config, maybe force yourself to apply those only with scripts or a config management tool like puppet or ansible.
But there are options to avoid online sync: If your system fits in 1TB or less, you could only carry a external 2,5" SSD (or maybe even just a USB key) around and boot all machines off that. Or even carry an entire NUC, they still are more compact than a laptop.
Then sync would no longer be necessary because all you would ever need to use is that 1 unified device.
Until then, I use network shares on a cloud server through VPN for media, and Git and Vagrant/Docker for dev stuff.
It more convenient than cloud storage in that I don't have to worry about sensitive information being stored on remote servers, but obviously is restricted by the storage capacity of each device.
Also, since it's better to err on the side of saying something irrelevant than not saying something at all, check out http://xpra.org/.
Almost all the ideas so far mention moving the whole thing with you (usb sticks with system or vm), or working remotely (X / mosh / similar).
There's another possibility if you don't require any (or only static-like) data to be migrated - use some system which can be updated / rebuilt in sync. Setup a server salt / chef / puppet and make sure your local machines sync to it periodically. Whenever you need some modification, you have to implement it in your server and push.
Advanced version - implement it over something like NixOS for consistent builds.
Pros: anything fails - just rebuild it (even remotely if you can set up the satellite servers to boot over network); you won't forget your physical media; you can update all machines at once; you can store all configs in a repo with full history; if you need per-site specialisations, templates are great for it
Cons: if you need to sync any large chunks of data, this is not a good idea anymore
The only drawback I could see is if the "implement [a modification] it in your server and push" would turn out to become a hassle to try out libraries and programs. But that's just me speaking without any hands-on experience with these tools. Thanks.
That's one way to quickly replicate your env across workstations. Then all you need is git/rsync/btsync + a few bash scripts to keep your data in sync.
I even have mulitple separate power adapters. I have one in my kitchen and another in my bedroom, for instance. They are literally 15 feet away. Totally worth the $70 or whatever since I can easily move back and forth multiple times a day.
Carrying around and dealing with a power adapter is 60 to 70% of the hassle of carrying a laptop and that can mostly be eliminated.
Oh, I also have a separate pair of headphones in each location AND one that always stays in my bookbag.
By encouraging everyone to work off network drives, everything tends to stay in sync. Other solutions include having a server-located user data folder (desktop, documents, etc. - but that slows down for regional sites or mobile data if you load it up too much) and horrors like sharepoint or other remote content storage programs (HP records manager, etc.).
For code, I run a git server on a network share and point visual studio at it. My code gets committed to it and I can see it from any device on the network, including via VPN.
I've tried using BTSync and syncthing (https://syncthing.net/) to automatically keep directories in sync with each other, but there's quite an overhead to have real-time change propagation and it doesn't seem to add much value if you aren't actually using the remote nodes at the same time. Instead I've reverted to using Unison (http://www.cis.upenn.edu/~bcpierce/unison/) when I've finished working in one location and want to push those changes to another node.
Actually, Macs are quite enjoyable in the sense that everything else is taken care of, by Apple - The App store handles all my software installs and purchases between computers, iTunes handles all my music collection and ensures all 3 Macs have the same tracks (I use Apple Music w/ Match) and iCloud makes sure all my docs are available from any device I use (iPad/iPhone/etc.)
As for browsing and bookmarks, I use Chrome.
LastPass and XMarks to keep passwords and bookmarks synced across both (and across browsers as I use both Chrome and Firefox).
Dropbox. In particular, I run Pidgin for my instant messaging (with skyp4pidgin) on and encfs encrypted fuse file system. (On the Windows side, I use Boxcrypter Classic for encfs.) I also use Dropbox to push repositories back and forth (I use Mercurial for that, but Git would be just as good.) If I didn’t need to use Dropbox for sharing some common folders with others, I would consider SpiderOak and then not worry about encfs.
On the work machine, I have a fairly large TrueCrypt file container that holds my Firefox profile and Chrome user data, Dropbox folder and any other data I don’t want other users of the computer to be able to access when I’m not there.
I can also ssh to my home machine from the office, which is handy if I’ve forgotten something. I also use it for running rsync and/or unison as appropriate one some data (got to keep that office music collection in sync!). Cygwin on Windows for rsync and unison clients (as well as a bash terminal.)
I can also ssh to linux machines at work from home, and use x2go to rdesktop/vnc to office machines when I need to occasionally. Have found this much faster than alternatives (rdesktop/vnc over VPN, or xpra).
Hope that gives some ideas of things you could try if a full VM image shuttling back and forth does not work for you.
You best bet is to run your workstation in a VM and put your data files on a mount to the underlying file system and then run Unison file sync across the two file sets (the VM image(s) and the files).
Putting all your files inside the VM will work as well - but copy times will be longer because the whole thing will be bigger.
With your bandwidth limitations anything magical is out of the question; live file sync is an attractive idea but it's too slow for large file sets - Dropbox, Lsyncd, Git Annex - all suffer when dealing with a lot of files across low bandwidth. Unison, on the other hand, works well with a USB key as the transport mechanism - and only runs when you ask it to - so it doesn't keep transferring files as you change them (which seems like a good idea - but is really a PITA when you're not actually sharing files with other people).
Going a bit fantastical here - but IMHO the gold standard is block level sync with DRDB - which will keep two or more systems identical at all times; but it's too heavyweight for your purposes.
More practically, just install the same software on both your workstations and sync your data and dot files (e.g. .home) with Unison.
This is what I have been doing for over 5 years - and it works really well.
Config-wise, I just built a shell script that I ran on both machines to install the same packages and libs I use for work.
For my startup, we use something like this https://github.com/thoughtbot/laptop (modified to our specs of course) to keep our machines all synced up. I'm thinking of using Docker or Vagrant to consistently provision machines with regular updates though. This has to be explored.
Does exactly what it says on the tin!
Minor issues I have experienced with this setup:
1. The biggest issue of all is, due to the peculiarity of USB power design, most SSD's might experience failures when the machine is powered off. This is a known issue with USB keys too! Believe me, booting a system every time not knowing if it's going to end in the dreaded fsck mess with thousands of errors really is a nightmare. Going with the Intel 530 series SSD's (which is a known solution) fixed the issue as they have larger capacitors, though Samsung SSD's might be reliable too.
2. USB power: on most laptops, your single USB power cord is unable to power the external drive; you might have to use the popular Y (double connector) USB cable.
3. Stay with open source video drivers. Especially swapping nVidia <-> AMD drivers is a recipe for disaster.
4. Audio HW devices will stack up, possibly resetting their settings when you return to a previous machine.
5. You might sometimes have issues with UEFI when you first boot your system on a new machine.
[6] VirtualBox VM's have sometimes issues when the system unexpectedly swaps eth0<->eth1 interface names, but this is easily fixed in the network configuration before booting them; also swapping AMD and Intel processors is risky, and can unexpectedly hang your Windows system on boot; BTW don't forget to carry along on your external drive every hard disk image they might use, or the VM's won't boot. Of course this applies to Vagrant boxes too, but is less relevant as you can destroy/recreate the box any time.
- GitHub/GitLab
- owncloud with personal "webspace" for data, especially important for Android (+ a photobucket account for images for random funny forum posts)
- Zotero for Firefox (with said owncloud) for my academic work. This is worth its weight in gold. I prefer to work with Latex but you are often forced to use Word/LO and the plugins are amazing (instant citation/bibliography; snapswitch to a different format).
- Thunderbird (and Outlook on OSX, meh but needed/convenient for work would prefer 100% TB)
- Firefox sync (not really using it anymore but probably should)
- KeePass to store my passwords across systems
The overlap between the machines isn't huge and I mostly just keep them up to date. The stuff I use on multiple machines is on the same version if possible but not automated in any way. My personal backup strategy is pretty horrible (I just randomly put important stuff on external storage), I need to up my game there. Not even keeping my configuration files etc. on github as of now (planned for the near future) tl;dr: Need better backups, need scripted os sync
Note: I don't use my phone all that much for stuff I can do on other machines (don't mobile browse a lot). It's mostly an IM/call platform with occasional data use.
I use tmux and then connect to the session from work, home, etc... and setup ssh config (http://nerderati.com/2011/03/17/simplify-your-life-with-an-s...) so I have shortcuts to all of my remote machines. App environments are built with docker, snapshotted, and pushed to dockerhub.
http://thenextweb.com/microsoft/2015/04/29/windows-10-will-l...
http://www.gizmodo.co.uk/2012/02/who-needs-a-pc-when-you-can...
Ubuntu Edge (no longer available)
It's idempotent, so whenever I need to make a config change, I just change that file and run "provision", and everything is the same across machines. I'm not sure if this answers all of your questions, but it's good enough for work (personal data can go on something like Dropbox or Syncthing).
git for code.
then i have VMs on EC2 for specific tasks.
>tired of carrying a laptop
i solved this with a lighter weight laptop.
you can usually leave your multiple displays wherever you need them, and plug into the laptop.
edit:
looking back, i'm very invested in google's ecosystem of "synced" things: google apps.
it's almost to a 90's microsoft extent -- where i will actively avoid and discourage incompatible formats or platforms.
at the very least you can use any of google's things with an OK computer and an internet connection.
that leaves me wondering what will "obsolete" the "google office"
- Outlook: MS Exchange does syncing on email/calendar/tasks (I use Nine for my mobile mail client).
- GDrive: General file storage/accessibility.
- Chrome browser: Sync's our tabs/history. I use Tab Cloud to save sessions across Chrome too.
- One Note: I use this for meeting notes and brain storming.
- Google Keep: I use this like sticky notes thats accessible cross devise.
With this mix I can easily move between my laptops/mobile devices as needed.
- Linux on DigitalOcean droplet through ssh/mosh+tmux (usage of CLI/TUI software and stable network connection is required)
- full Linux system on USB 3.0 flash drive (reliable backup configuration is required)
Now I use similar Linux desktops on each desktop/laptop with:
- notes syncing using unison
- mail stored on mail server and accessed through IMAP client (mutt)
- dotfiles managed with git repository
- projects managed with various VCSes
- sensible data managed with pass/gpg
You sound like you would do well with that idea, but maybe it takes the form of an external (e.g. USB) storage device that you store virtual machines on but don't forget about backups. Perhaps you have a script snapshot and rsync diffs between your storage and your machine so that each machine's local storage can also be your backup.
None of it ever worked well so I bought a nice leather messenger bag and learned to like carrying my laptop around and using the small (13") screen most of the time.
Not perfect but it is ok.