But having your home server hacked and then presumably your entire home network and everything in it - seems way too fraught to even attempt it.
Thoughts on that? Am I just too unfamiliar with network security and this actually solved now — and there is already a well-defined trusted approach to this?
The mistake many make at the start is trying to run a complex web application backend with php or databases or whatever. Or using some "easy" container all-in-one containing these complexities. Maintaining the security of that is a neverending diffcult task. Whereas maintaining nginx installed from your OS repos literally requires no work at all. KISS and you'll be perfectly safe.
Historically the security of Ethernet, IEEE802.11 and other such protocols has been full of half measures, laughably weak crypto and whatever WPS is supposed to be. Look at the history of wireless security if you want to have a good laugh.
In the application layer, on the other hand, we have rock solid solutions like SSH which remain the gold standard for security.
Yeah you can lock everything down, _if_ you're careful, _if_ you don't mess up, _if_ some consumer hardware doesn't have a vulnerability
Is separate physical hosts a real improvement in security? It seems like a real air gap vs relying on linux hardening. Lots of raspberry pis (something cheaper now?) vs one larger home server hosting multiple services.
My consumer router has a dmz mode, but I'm not sure how far I can trust it. I guess it's a good thing nobody uses any of the stuff I host.
YouTube channels get hacked so often, even from technical people like Linus Tech Tips.
As self-hosters we are not going to change the face of the world. The other 98% of the general public is going to use hegemonic services: self-hosting is a privilege for those who have the education, time and money to put into it. We’re only deploying solutions that work for us, individually.People are absolutely getting sick of subscriptions. It's also getting easier to self host. Tailscale has been a game changer for me personally as I just had no confidence in getting my services working correctly over the internet without getting pwned
And then after poking around for a week, they go back to Twitter and Reddit.
It is harder (but not impossible, and not without it's own inconvenience) for mastodon.social to do a rug pull because there are near-identical alternatives that others (or yourself) host.
"Pricing is based on the specification of the VM you are connecting:
$5 per vCPU + $2.5 per GB of Ram + $0.25 per 10GB of disk.
Example: if you are connecting a VM with 2 vCPU + 4GB Ram + 40GB Disk, the cost per month will be:
(52) + (2.54) + (0.25*4) = ~$21/mo
Price per hour is then calculated like this: $21/730h = $0.02876/h
You can create one BYOVM service for free. To be eligible the VM you connect must have no more than 2 vCPU, max 4 GB of ram and max 80 GB of storage." https://docs.elest.io/books/cloud-providers/page/byovm-bring...
Could you share how you think you compare to cloudron? Are you kind of a IaaS host coordinator?
I myself run a Next.js app on it, it’s amazing how much faster it is than Vercel… even on the cheapest VM (1 CPU / 2 Gb RAM) I now get instant page loads vs. long long seconds of waiting with Vercel.
Oof, good advice. I run a startup that helps folks self-host, but it really does split the audience in two. Folks technical enough to swallow the somewhat rough edges become huge fans and part of a fun community. Folks just on the other side of that split tend to have pretty frustrating experiences...
I dearly wish I had the capital to be able to spend another full-time year on making our product better, but self-hosting is a really tricky thing to build a company around - the audience by definition is looking to avoid paying for services!
I do still fully believe (and hope!) that one day, far from now, self-hosting reliably will be trivial, and our kids will all think we were a bit slow for relying on a few megacorporations hosted services.
Another huge problem is that there's a home network between your product and the user's other devices; most home networks are utter crap, and often even tech-savvy people don't have a whole lot of control over it (I hate my ISP's modem with passion). This seriously limits your potential to provide an excellent UX; IMHO it's the UX that makes or breaks a product for "the rest of us".
I used to self-host a whole bunch of things on a VPS, including my blog, git repos, a DIY blogroll / RSS reader, etc. In the end I've decided it was not worth the effort; the blog was moved to Netlify, repos to Github, and the RSS kludge got swapped for NetNewsWire with iCloud sync. I was paying €5 for the VPS, yet now I'm paying Apple €20 to host my email, sync my photos, get access to the music catalogue, etc. I would definitely pay €20/mo for a box under my desk + an online service, provided it gives me similar value without much additional effort.
I think the problem that KubeSail/PiBox is aiming to solve might be both too broad (run any software you like!), and too narrow (if you're an enthusiast!) at the same time. I don't want to run Miniflux; I want to have my RSS feeds synced between devices. The software that pushes the bytes (and the hardware it runs on) should be invisible - unless I decide (out of my own free will / curiosity) to pop the cover open and start tinkering.
I don't think you can solve this by addressing shortcomings in a single piece of the stack. Both the layer below you (your average home network), and above you (the apps) have their own problems; some are like splinters (tiny but enough to ruin the experience), some are fundamental ("what is MySQL and why do I need to know"). I don't think it's a lost fight, but I would try to start with a vision for a more vertically integrated solution; maybe one step of that road is to eventually build your own WiFi AP/router (or even become an ISP), maybe to make a deal with Spotify (or even directly with EMI/WB/etc)... I don't think a task is too big if you can seriously challenge Apple/Amazon/Google at the end of the road.
I've spoken with several people who are starting similar companies and who've reached out to me (happy to do that!) - my advice is similar to yours: keep it simple, keep it focused. KubeSail is a developer tool turned home-hosting tool, but if I could rebuild it, I'd make it incredibly simple to get Jellyfin and a torrent/VPN client installed and that's about it, and then execute insanely hard on making that as streamlined and foolproof as humanly possible.
What is their modem doing that you haven't been able to work around?
Our best feature is that the website will detect if you're on the same network as your machine and if so, offer "local" links instead of remotely proxied ones. That way non-technical users dont need anything fancy or to be aware of how NAT traversal works. On top of that, the "local" urls still get valid HTTPS certs for free, so non-technical users dont get any scary browser warnings.
We started out as a way to make self-hosting easier for corporations, and were doing consulting work, but the users who joined our community were mostly home-hosters, so we leaned into that! Jellyfin is now our most popular app.
First, blockchains are terribly limited capability-wise. You'd be much better off with a raspberry pi.
Second, there's no such thing as "no one controls". There's always control. Somebody is at the top of every blockchain in existence, and their interest probably doesn't align with your.
Eg, Ethereum being expensive is a problem for the users, but the people who get paid the fee love it, so there's no reason for them to be interested in decreasing costs.
who is paying for this?
Proof of waste is a colossal externalised cost; you think you're trading "your" electricity and dollars for "your" imaginary money; but the fact is, you're wasting my planet. Cryptocurrencies have already caused enormous harm, and even as the fad is waning, it couldn't die soon enough.
I have a server in my basement with like 35tb of zfs storage to hold my blu-ray rips. The movies are backed up onto tapes, and those are more or less durable but not really readily-accessible (and kind of a pain).
A very large quantity of my time is spent mucking around with disks, and fixing data issues. Even when there's no data issues, there might be a transient read error which causes a fault and I have to spend time dealing with scrubs or at the very least checksumming files to make sure that they're fine.
A masochistic part of me kind of enjoys it, but honestly it's gotten to a point where I'm debating just paying some money to Hetzner or Amazon and selling off the servers.
Episodes appear to be stored in no particular order on Blu-ray, so I end up having to open the video file, pray for a title card, and match that against an episode order list on Wikipedia.
For some shows (like most British shows) this isn't too bad, since there's a very small number of episodes so re-ripping doesn't take too long. For other shows (e.g. Adventure Time), there's a million episodes, and correctly labeling them takes a lot of effort that I do not want to duplicate.
The thing is...I have re-ripped all my blu-rays. Twice. Because I didn't know what I was doing with ZFS and kept breakingu cluster. I don't really want to do it again.
That would give the user the same experience as with a desktop application. Full control over their data, saved locally.
The problem is that, according my tests, Firefox does not support it at all. Chrome does not support it on Android and Safari does not support it on iOS. Not sure about Safari on the desktop.
Here is a text editor demo which let's you try if it works with your browser:
https://googlechromelabs.github.io/text-editor/
If your browser supports it, it will let you load and save files just like a desktop application. If it does not support it, it will use a download/upload workaround.
If you're not familiar it's a file-system like API for writing files to an opaque non-user-accessable file system. Your application could probably provide it's own export functionality using blob urls, and import using traditional file "upload".
I just tried the text editor example in Firefox and it works fine for me, although all the newlines in my file were ignored so it looks like garbage. Maybe it assumes Windows-style line endings?
EDIT: Oh, no, it just doesn't support line endings at all? Even if I press the enter key I just get a space. Maybe it's just a proof of concept and not an actual working text editor.
But it only addresses half of the value of self-hosting (which is much better than nothing). The other half is: being able to have control over the software itself, when/if it gets updated, being able to be sure what's done with the data (if you're sufficiently motivated), and not having the service become unavailable when the internet is out.
• Knowing when disk space, inode usage, or memory usage get high, long before it’s an emergency.
• Automated monitoring of SSL certificate expiration dates, letting you know days before a certificate expires. Whether or not you use something like certbot, have a separate process that automatically tells you a certificate is close to expiration.
• Automated periodic end-to-end testing of moving parts. Like if you run an email server, a process that sends something from your server to a gmail.com address, and then checks the gmail.com inbox to find the message.
• Automated periodic testing that unexposed ports remain unavailable from outside the device or private network.
• Automated checking that a Linux instance is successfully checking for and installing security updates, and is not waiting for a reboot. • Automated checking that backups are working as expected. You might not be able to automate periodic restore testing, but at least check that backups do not appear to be silently failing. • Separating out low priority alerts from high priority alerts. You want to get woken up when necessary, but not for an issue that can wait until you are at your desk.
Benefits include:
- Security
- Ease of configuring traffic control: As long as you're not redirecting UDP (have fun lol), steering apps with HTTP or SOCKS5 forward-proxies is so much more straightforward than routing.
- Performance/effieciency (global package cache for your network!)
- Resilience (apt upgrades and docker image pulls can keep working despite your entire network being offline)
My rough starting kit for a Linux-based network here would be:
- Some caching forwarding internal DNS server. If you already have an internal recursor or forwarder great, but it's good to let the DNS server serving your clients be separate anyway. dnsmasq/unbound/technitium/coredns/powerdns/yadifa.
- Internal NTP for syncing time. May be provided by your DNS or DHCP server already. chrony is good.
- apt-cacher-ng or other caching forward HTTP proxy for your apt/dnf/pacman/apk/whathaveyou updates.
- docker-registry-server in mirror mode and set up as mirror for any docker/podman hosts you have.
It runs on a Linode instance with a webapp whose sole responsibility is to respond to Pingdom requests. There are two URLs that Pingdom looks for: one that returns a 500 if the JSON file indicates an issue that warrants texting me. A second that returns a 500 if the JSON file indicates an issue that warrants emailing me for a lower priority issue. Pingdom is configured accordingly.
If for any reason the JSON file has not been written in the past 10 minutes (?) or cannot be read and parsed, both URLs return a 500.
The script has a log file, so when I get an alert I can check the log file to determine what is wrong.
This is likely atypical, but it works really well for me. My scripts do the work of monitoring the heck out of everything. I only need Pingdom (or a service like it) to monitor two URLs and do the texting/emailing.
But my overall approach is to think of monitoring like unit tests or integration tests: when I think of something that could go wrong, I try to make sure there is monitoring that can detect it and alert me. When possible, before it becomes urgent. And when something does go wrong that is not automatically detected, it's a high priority to add monitoring around that.
apt-get install postgresql, connect it, and you're done.
It doesn't seem that simple. When I researched Nextcloud in the past, I avoided it because of warnings like the ones in this thread: https://news.ycombinator.com/item?id=25481465
Ctrl+F search that thread for "failure".
If Nextcloud has solved whatever issues were happening in 2020, it still doesn't necessarily instill confidence because one can remain skeptical and assume there are new issues still happening in 2023. E.g. https://github.com/nextcloud/server/issues
It's going to take some time to wade through all those Github issues to determine if there are any showstoppers that would affect one's installation. This doesn't look like a low-maintenance solution. The gp's wording of "dedicated the time" seems very relevant. Copy&paste of some YAML doesn't really address the work involved.
Google wants me to use them, but they have earn my lack of trust - between deprecating services that look useful, the algorithm locking a few people out with no way to get back in, random changes that make useful workflows break I'm not interested.
Feel free to contact me for a totally not shady hosting :)
Not shady is important. As is reason to believe that you are contributing to make the software you host better (as opposed to mooching off free software)
I ought to be able to choose a person that I trust to not lose or leak data, a different person to curate code for use on my device, and yet another person for being authoritative about the problem domain.
If I later decide that Jimbo has bad taste in client side code, I shouldn't have to also abandon Mary's excellent data handling track record. Yet somehow we've found our way to a corner of the design space where each entity that carries any of these burdens also carries the others.
It's the user's trust preference that I want to matter, not the app developer's.
My NAS appliance has plenty of storage available, I want to select it as my storage backend so that if the internet goes down everything that doesn't require collaboration still works for me. And not because I've been very choosy about what code I rely on, but because that degree of composability is built into the protocols.
I am paying for hosting, I would have to spend time to make backups on my own, now someone with experience is doing backups for me.
It is just like I can change oil in my car on my own if I go to the shop they are not going to do it for free because I can do it on my own.
"SSO tax" is also because someone has to spend time to set it up and maintain for specific company. We slowly get to OAuth2 everywhere and Azure Active Directory/Other providers where it won't be a hassle but still bunch of big companies keep on their outdated SAML services thinking it is secure and they would like you do do their job.
I’m not a security expert but it makes me feel like keeping software up to date is less urgent. That lets me stick to one version for a while once it does everything I like. The stability of experience and ease of use is greet.
It enables a kind of bring your own account (BYOA?) installation process. Where self-hostable services would be entirely built based on managed services.
- Infrastructure as code. The installer takes in any <cloud_vendor> account and provisions + configures the required components
- High availability built in
- no need to support old or niche hardware
- On-demand costs structure. Many self-hosted services don't need to run 24/7
My biggest fear with raspberry pi or VPS is the security. But self-hosting does not mean my-server-hosting. Some amount of vendor lock-in is acceptable and using the same APIs and processes as enterprise users sounds like a win. At least compared to not self-hosting at all.
Of course many things are still missing:
- self-hosted tools that actually work like this
- connection between data center and home. To integrate with smart home/IoT and similar things
- a reliable billing model for less technical users. It has to be impossible to rack up huge cloud bills
For now I guess it's just not yet mature enough. But I would like to see the serverless mentality finding it's way into self-hosted software communities.
* https://news.ycombinator.com/item?id=36986980
An example of what I mean: https://github.com/full-stack-serverless/conference-app-in-a...
I don't see any reason why that shouldn't also work for more typical self-hosted applications
The second is having to read and learn provider specific documentation is a waste of time (ie deploying on fly/supabase/heroku/netlify, which all have their own cli tools and their own config syntax)
You’re still writing software for others to use, but you don’t take responsibility for their uptime or content.
It’s a little bit of a barrier because you need to create two free accounts (including GitHub) and learn your way around. Part of open source in practice is education and I think teaching people enough so they can edit a file on GitHub would be empowering, even if that’s as far as they go.
Those are services I’ve used that have a free tier and seem pretty low-maintenance. What would be other good choices for this sort of thing?
As someone who self-hosts a good chunk of the services that I use, I am in total agreement of the challenges that face the poor souls that seek alternatives to big tech.
From Oracle (allegedly) randomly shutting down instances [0] to Google doing A-B testing [1] on how to further lock down Youtube videos.
It truly is a treacherous journey for the self-hosters.
[0] https://gist.github.com/yewtudotbe/c16a69ddad88a37c2a364a5ff...
[1] https://github.com/iv-org/invidious/issues/4027#issuecomment...
At this point it's used by more than just me, a bunch of people in my circle use my instance to share files.
In case anyone else finds this useful: https://github.com/aaviator42/izi
There's a demo here: https://aavi.xyz/proj/fakeizi/
- the development and use of services useful at small scale is essentially ceased in the last decades, meaning it's harder to keep up. We still have emails (even if current antispam solutions makes hard to have personal mailserver able to communicate with anyone) but feeds are more and more useless since most sites or do not offer them or publish just titles and ads and so on;
- older services got abandoned and modern ones try to mimic the giants ones, being needlessly complex and heavy for personal use.
Let's talk clear:
- we do not have modern MUAs, comfy enough. Yes, we have notmuch-emacs, Mu4E, but a proper setup demands few hundred SLoC at least, not something as simple a state: this is the root dir to downloads all my messages, keep them on server or delete, few filters and auto-refile rules, remote credentials and stop;
- we do not have file sharing stuff the easy way, the least obscene is WebDAV that's supported by most OSes, but most people do not know it, so we just need web-apps to mimick a file manager Google Drive alike to makes others able to reach our files;
- we do lost most of the desktop computing model, with people on limited and limiting mobile devices, who happen to be integrated only with cloud crap;
- IPv6 is not that widespread in the form a a global per any device, and personal domains are not much used by most.
Technically ANYTHING needed is there, but since most people do not know it and some bi&powerful want anybody on their servers we essentially have very little margin of maneuvers.
Modern telephony is old classic VoIP, but most carriers do not offer few settings to connect any softphone or a personal PBX (Yate/Asterisk) to them, mails are still there, but for most mails means webmails, some big vendors have even buggy IMAP (GMail) or no IMAP/POP at all (TutaNota) or try to push their new favorite protocol (Proton Mail/JMAP). The value of having messages managed on personal iron, locally indexed, having a domain name with various subdomains and so on is unknown to most. Cars nowadays have wifi and mobile connections but nothing to be directly connected to their formal owner, anything goes through the OEM server, who happen to be the substantial owner.
In the 2030 "you'll own nothing" is a THREAT TO THE HUMANITY but most seems to like it and few like the profitable outcome of that. That's the real issue.
Neither application has a built-in mechanism for backing up your application data that is anywhere near user-friendly.
TrueCharts does not have the GUI backup mechanism besides to protect user data per application. One application that becomes corrupt? Good luck, you have to restore all applications to a single point in time if you used with their commandline tool.
Unraid does not have any sort of backup mechanism relies on a community for setting up backups.
ZFS replication is not enough special care must be given for applications to be in their correct state to prevent data corruption.
I think docker has made this a lot easier than it was and the new NAS operating systems making deploying common popular containers really easy so its more accessible than it once was.
Let's say you are running Ubuntu 20 with nginx simply hosting some static websites, and you let it run for 5 years without any updates.
Are there vulnerabilities so big found with the OS or very popular software (like nginx) that they could compromise your server and give root access?
I’ve written 3 of them.
We’re unlikely to self-host, but we’ll almost certainly be doing some kind of cloud service for them.
Thankfully, the scale is minuscule, compared to what a lot of folks, hereabouts, are used to.