The server is a Dell Precision T5610 (20 physical cores, 64GB memory, 4 Samsung Pro SSDs and quad gigabit NIC additionally to the internal one) and a QNap TS-531X with Seagate IronWolf drives. I would recommend against using PIs at this point, see other comment[2].
Right now all services are running in docker in a dedicated VM on VMWare Workstation. The server also hosts a full work environment I can RDP and SSH into, depending on what I do, when I am traveling only with my iPad Pro (RDP because my last job involved a lot of Java for which I prefer IntelliJ). At some point I should likely migrate to vSphere, but I had the workstation license already and it was quick to setup.
[1] https://screamingatmyscreen.com/2018/7/building-our-home-and... [2] https://news.ycombinator.com/item?id=17966264
Each Docker image runs it's own stack, and each stack has the potential to contain vulnerabilities. Even with services such as Watchtower ( https://hub.docker.com/r/v2tec/watchtower/ ), you're still not safe. Some images are abandoned for years, others are only updated once their "final" product is updated, meaning you could be hosting a handful of vulnerable services without even knowing it.
I host all my internet facing stuff on FreeBSD in jails, though Linux with LXC would do just as good. I have one stack to update, and once that's updated, everything else is updated as well.
I got out of social media in early 2012. Deleted FB and Twitter. I do maintain a presence on LinkedIn, but I never post anything there, nor do I login. It's mostly there because employers expect a LI profile. It's just a bland copy of my resume.
I moved my email from Gmail to my own domain from 2013-2015. It took almost two years because I had Gmail since April 2004, a few days after launch. My whole digital life was tied into Gmail, and moving stuff over takes time. I still have that Gmail account, but again, I never log into anything Google, and it simply forwards any email to my real account. I get maybe 1 email a month from Gmail.
I host my email with Fastmail, but I've also selfhosted, and tried O365. It's easy to switch, and I keep an offline backup of everything.
Now, my hardware and software setup.
At home, I have a Synology NAS with ~12 TB of usable storage. This is my primary data store - I have software that automiatically backs up all photos from my Iphone, Time Machine, Veeam, Rysnc, and email backup.
I own my own physical server (Dell R720XD) which is colocated in a datacenter in a different country. This server has 96 GB of RAM, 16 TB of storage, and 12 Xeon cores. It usually runs between 12-25 VMs, some of which are production (personal), hosting my websites, git, various projects, and so on. It's connected to the internet through a gigabit connection with a 10 TB monthly bandwidth cap (I never come close to 1 TB!). This costs me ~$85 a month. I don't do BGP, though I could if I paid some more and put in a proper router and got my own IP range.. as it stands, the 8 public IPs I get are enough for me.
The colo server is connected via an always-on, site-to-site IPSEC VPN to my home LAN (through my EdgeRouter X). I have symmetric gigabit, so it's practically like having the machine in my house.
My NAS backs up to the colo server, and the colo server to the NAS (via rsync). The NAS also backs up to a local USB drive, for quick restores.
All my home data lives in four places
1. The local PC/laptop/phone. 2. The NAS 3. The USB Disk 4. The offsite colo server.
Likewise, all my colo data lives in the same four places.
I can VPN in to the colo server and get connected to my home lan from anywhere in the world. Any sensitive data is encrypted - and anything really sensitive is not stored on a computer.
This setup works very well for me and my family, and I control all my data and am not dependent on a cloud provider. I also have a bunch of other servers running in my home, but they aren't part of my core infrastructure.
The cost?
~$1200 for the R720. ~$1000 for the NAS. and ~$85 a month for colocation costs.
What's strange and foreign to me is the world where people rely on other companies for so much of their online lives.
One issue I've found with a similar approach is that many mobile apps are limited to iCloud, Dropbox and Google Driver for state synchronization. Only a few apps (e.g. https://2doapp.com, https://notebooksapp.com, https://www.omnigroup.com/omnifocus/) make the effort to support open standards like WebDAV and CalDAV.
iOS now supports storage providers, so you can use a dedicated application to transfer files between devices via SSH/SMB/etc, but many apps cannot export/import all state via a single backup file.
> Any sensitive data is encrypted
Is this done on clients (E2E) or server? If the server, how do you manage encryption keys?
Is it possible to get "safe enough" NAS devices at this time?
* How do you backup Fastmail?
* How'd you find your colo provider?
* If it's in a different country, how does the latency affect you?
Route everything on the phone through a personal platform. Gain control of location, IP, maps caching, etc.
Centralize at a personal level.
What would your personal platform do? How would it take that data and turn it into something useful while also removing the privacy concerns?
Best you can do is have something that blocks or spoofs that data before sending it to the "collector". So you gain the privacy and you lose on the service improvement. If this is legitimate concern you can try to drop the most intrusive platforms from your digital life. These would most likely be Google and Facebook.
Alternatively there are options for you to use with more limited effect: ad and tracker blockers, secure or anonymous browsers, VPN, stricter privacy settings on your devices, etc. The end result is usually the same: the less data you provide them, the less useful their service is. Which makes trading your info even less enticing.
Google's search results are on steady decline over past years, result filled with ripoff pages, often original pages are buried deep in the list full of ripoffs. Started using duckduckgo much more because of this. MS drivers tailoring or whatever you meant are presumably a joke, I know which driver for which hardware I need and if not simple system scan should be enough. And FB feed is crap too, worst point was when feed was completely randomized on every reload, very frustrating. Even now its pretty bad - some posts hang on top for a week, interesting new stuff is buried somewhere deep.
I would be much happier without any of those. Good effect is, these 'platforms' are getting worse and I am using them less and less (FB is at the end anything but social), or using competition
I really like the idea of having centralization at a personal level, but it seems like the most you can get out of it is fairly hassle-free synchronization of your stuff between devices. Which is nice, but I don't see how it could provide Google-style features that are useful mainly because of the aggregation of many gigabytes of user data per day.
These two are kind of questionable; I'm not convinced that the personalisation gives me better search results and I use FB exclusively to communicate with people I know.
What they really give you in exchange for monetising you is that they do the sysadmin work for you. That's not nothing and for the average user is really quite a benefit. It's just that people vastly prefer a zero-cost to even a low-cost provider, because it's easy to get started and harder to switch.
Given that there are finally good non-google options for a lot of these (OSM + Thunderforest + Graphhopper gives a really nice map experience for instance) I'd love to see it go further. As always, it's hard to find time to work on this though.
Initially i went the Raspberry Pi route, and in a few months i had 4 or 5 of them running, which was where it started turning into a chore. While i enjoy tinkering, i don't particularly enjoy keeping a host of machine up to date.
Fast forward a couple of years, and my home setup now looks like this :
- UBNT EdgeRouter for that sweet affordable hardware L2TP/IPSec VPN.
- Intel NUC6, Running everything internet facing in FreeBSD Jails.
- Intel NUC7, Running everything internally available through Docker, along with Time Machine backups.
- Synology DS716+, Holds all data from both NUC machines, as well as personal files (documents/photos/music/movies)
Resilio Sync is running on the external NUC to provide an "always on" node.
I have a couple of ODroid HC1 boxes that holds my backups, one at home and one at a remote location.
The only thing i have yet to migrate is mail, which is still located at GSuite. I've moved several times, but every time i decide it's not worth the effort. Since "everybody" uses GMail every mail i send will eventually get indexed as well, regardless of where i store my own mail. I use GSuite for hosting 5 domains, and i've yet to find an alternative that doesn't require either lots of money or lots of administration from me.
Also, i remember from running my own mailserver 5-8 years ago, that keeping it off various spam blacklists is a job in itself.
The recent Intel scares have got me interested in firmware and Libreboot again, and a HN commenter[0] pointed me at some extremely beefy (e.g. 16 cores!) AMD motherboard/CPU combos, that it’s possible to run fully-free firmware on.
Without doing much actual research, Xen on a big box seems like it could be a good way to have a single physical machine where I ‘deploy’ different services in VMs.
Anyone gone this route, or can think of any problems with this approach? It seems like it could be a little easier to manage, but I’m sure there are extra considerations for networking and security.
As long as only a few users (in my case between 1 and 6) use the services you'd be surprised with how little resources you can get away.
My initial test VM (VMWare) I used had 1 CPU and 1GB memory - it ran Postgres, miniflux, docker registry, gitea and wallabag without problems. CPU mostly idle, memory without a lot of load ~350MB. (Alpine as base for docker)
And of course you want redundancy against losing an entire machine (say the PSU fails and overvolts the 12v rail). But more centralized ad-hoc infrastructure should be easier to grok/manage/backup against this.
My new router is pfsense running on KVM/Debian on an i5-4430. If I start running more Internet-facing services from home, they'll probably be on that box rather than my Libreboot KGPE.
FWIW, Newegg has been clearing out those KGPE boards, occasionally selling them for $80 - https://pcpartpicker.com/product/yvJwrH/asus-motherboard-kgp... (the last one was on flash.newegg.com and so doesn't show up on PCPP).
Non-high-density G34 heatsinks are a bit tough to find these days, but there's a thread where someone details the steps of mounting a 212 Evo. The Raijintek Aidos worked for me just by fashioning the appropriate length of steel bar, drilling four holes (two for the G34 mount, two for securing the existing Raijintek piece), and adding the appropriate screws/springs. The fan mounting on the Aidos does suck though.
I think the main factors that killed it for me, that made me drink the Kool-Aid so to speak, were a combination of getting an Android phone and the shutdown of Lavabit (with all the hassle that incurred - notifying dozens of colleagues, mailing lists, etc of the change). Concurrently with this I had just gotten my first industry gig at a pretty large networking equipment company, on a team with mostly older, mellowed, senior engineers for whom programming was "just a job". I didn't want to seem like a "paranoid weirdo" who had some black-hat alter ego. I sold out.
Since, I've pretty much moved wholesale to Google. I still don't use any other digital services - Google has basically become my one-stop shop, for better or worse. I use Android, Chrome, Gmail, Drive, Music, Books, Search, Maps, Keep, Photos, basically the whole damn suite. It's a beautifully unified and seamless experience. I feel in-general, Google gets just about everything right (I don't use Docs - I still write docs in LaTex, haha). It's quite a 180 from what I had before.
But ultimately, I think this has caused me a lot of cognitive dissonance. I've spent a lot of time thinking about how to "get back" lately, but this is tempered by how much control I've already given up and, well, what is frankly a pretty damn high quality and convenient experience and there are some things (like Maps, and Photos) which I really don't want to give up. I also don't hold any delusions that anything I do is going to be "more secure" from any threat model, really. I guess I just miss all the DIY. The creativity and control.
Anyways, recently I've been building an ARM64-based "mini-homelab" around an Archer AC1750 router running OpenWRT, a stack of three Odroid C2 SBCs, and an Nvidia Jetson TX1 (with a 50K LUT FPGA on the m.2 PCIe slot). I also have an ARM64 VM in the cloud. Once that's all set up, I've been considering how much I can "get back" under my control.
I'd had two google accounts, one created a decade ago on YouTube and later connected to my gmail-google account. I used the same login for both, and could switch between them without problem. I had used my Youtube-persona for most of my google docs work. That was all fine until suddenly I could only access docs from my other gmail-persona. They had without warning or reason changed either the account type or the app permissions for my account type. Not a word of warning, or even a message. There's no reverting it, no one at Google has been able to undo the change or recover my documents.
NEVER AGAIN will I trust any important information to any cloud company.
That was the day I started to take everything off the cloud and access it through a NAS at home (with off-site backup of course). I've never slept better
People think of backups as a way to protect themselves against hardware failure, but that's a reductive view; their purpose is to protect against systems failure, and a company is a single system. Sure they have their backups, but as you experienced, from the outside this is irrelevant - the system can fail as a whole, and therefore should be treated as a single copy.
Following the 3-2-1 rule, that might mean creating a second Google account with which you share all documents, and some process for backing up to outside Google (even if it's a regular manual use of Takeout).
Using a cloud service that can't be backed up is no different than trusting an hard drive not to fail.
I always like to read experiences and opinions of people who lean one way or the other with regard to all this stuff, or are at least cognizant of it at all, since it seems 99% of people just use whatever they happen upon first and solves their problem.
Also, if you're paranoid, your data is more exposed. You can turn off all your devices with disk encryption when they're out of your control (usually) but if you turn off your NAS while your away from home it's useless. And if it's on, physical access, and therefore your data, is easy to obtain by the moderately motivated.
Google is like a feudal lord: they might own you, but they'll protect you from everyone else weaker than them.
I keep everything except the boot drives on encrypted drives, so that in case of burglary no data is readable. The boot drives hold no data or passwords, only enough to start up and allow SSH logins. It's a small chore to login and manually mount the drives, but IMO worth it.
As for physical access, besides the 40kg German Shepherd Dog roaming my house, the same rules apply to access as from the outside: 2FA, and limited login attempts. I do expose more services on the LAN than i do on the internet, but everything requires authentication.
For personal cloud stuff i use Resilio Sync. It's not dependent on a single machine being powered on, and i have a couple of machines at different physical locations (both _mine_, as in hardware and sysadm tasks) "hosting" the data.
I can also power down system when leaving for a longer period and simply WoL them once connected to the VPN.
I’m interested in self-hosting for autonomy more than security.
That said, the data Google controls will likely outlive Google the legal entity/business-model of today. Including that huge security apparatus. The long-term (say 10 years from now) security of your data is moot if you can’t control it.
Google have been quite good with allowing exports, and promising to delete things (probably more like perma-hash, but I’ll take it). However, they’re increasingly behaving in ways I don’t expect or like. It’s been great, but they no longer seem like good stewards to me.
Getting NSLed or TOSed is a potential risk, not a guarantee. For most people who are doing nothing controversial or interesting and who are just using the cloud to receive service notifications and correspondence from friends (who are also doing nothing controversial or interesting), it isn’t a loss of control.
This is a real trade-off, and neither is right. But don’t pretend that keeping a half dozen complex services up, backed up, and secured isn’t a huge time investment. You can’t get that back.
Of course it is not feasible for most people to do this, just like it's not feasible for me personally to rebuild my car's engine, or grow vegetables in my garden. Both of which my neighbor does with great ease.
The point is, I use my skills to make my life better, as everyone does. I enjoy it, and it has the side effect of keeping my data private. For things I'm not good at, I either spend time getting good, or I pay a professional for his time and skills.
My mother uses Google and Apple services to back up her photos and data, and I wouldn't dream of forcing my approach on her.
My spouse asked me to set all this up for them, since they share my feelings about data privacy.
For most services I picked SQLite, so the whole backup is one file / directory. Software can easily be updated or test instances started since all is driven by docker. For the few services that need a DB there is a central one serving all hosts. Backups regularly (on a nightly basis where sufficient) go to the NAS. The VM docker is running on gets regular snapshots. Some initial setup and that's it. The whole setup is as movable and reproducible as I could justify making it over a weekend.
Securing theres is an interesting point. I take the risk of a local attacker. Other than that all I got to keep secure is the VPN, which I would have to do anyway.
I am not trying to make it sound like this is something you spend 10 minutes on and call it a day. I am doing this (running prod services) to various degrees for nearly two decades. If you are just getting into all of this, sure, it will take more time and dedication.
I see your point, but I do not necessarily agree with it. I would argue that even with "nothing controversial or interesting" it still is a loss of control. You do not control your data. You do not control if the service will stay available to you and / or your friends.
Hopefully we're heading towards a future where any non-tech person can easily setup a "personal cloud" by buying a commodity plug-in device, and have secure remote backups at multiple friends' places with a few clicks. Alas, money to actually fund software development was the real win of the surveillance industry.
But to echo your comment - the problem isn't really the time spent administering your own infrastructure, but rather when circumstances prevent you from spending that (focused) time.
I use Nextcloud for this, every picture I take is directly synced to my own basement. I can browse them and share them directly from the ui. Nextcloud also offers a nice webui for your email and allows syncing of you calendar and contacts from iPhone or Android. For me it allowed me to switch away from Google for my phone's back-end.
The one PI I still run as build server operates of a write protected SD card and reboots regularly.
I am not 100% sure why the PIs crashed yet (still have to do the investigation), but if errors like this already show up in a server rack with AC attached I'm a bit skeptical.
I've also run a large part of my internal network on RPis, but have since moved to Intel NUC machines, where i replaced a bunch of Pis with 2 NUCs. Storage like you has always been handled on a NAS. Synology in my case.
I don't have need for much computing power, so the NUCs handle that just fine.
See my other comment : https://news.ycombinator.com/item?id=17966507
For the smart home setup I am mostly using Elgato Eve SmartPlugs, their door and window sensors and Homebridge for everything that does not integrate with HomeKit natively. Any light source I cannot directly plug into a power outlet (ceiling mounted e.x.) is at least remote controllable via RF, not IR, and I am currently working on figuring out how to send the control signals through an PI or Arduino with an RF transceiver.
[1] https://news.ycombinator.com/item?id=17966311 [2] https://github.com/nfarina/homebridge
Here's a bash script + the set up I'm running to do that: https://nickjanetakis.com/blog/automatic-offline-file-backup...
