Opensource Linux-vserver https://en.wikipedia.org/wiki/Linux-VServer was available in 2001. At the same era commercial Virtuozzo was ubiquitous amount hosting providers.
Back in the earlier days of containerisation Linux had no options (Linux was pretty late to that particular game) and Solaris wasn’t free. So FreeBSD made a lot of sense.
These days the tooling around Linux is better and there are open source forks of Solaris so FreeBSD might seem like an odd choice for some. However I still think FreeBSD is a rock solid operating system and one that doesn’t get taken as seriously these days as it should do.
No other OS gives me this kind of comfort and stability.
* lower runtime overhead
* quicker deployment time
* smaller image foot print
Plus most of the advantages of VMs can still be applied.
However it does massively depend on individual use cases. Qubes aims to be ultra secure and containers in Linux weren’t up to that task at the time (the situation has since improved massively).
Tells me you don't know the difference between HW-Virtualisation and OS-Virtualisation.
For instance, awk can act differently and if you don't know the 25 year old decisions that led to the differences, it can be very confusing. (There are different behaviors and command line switches between GNU's AWK and the version from SVR4/XPG)
[...] In fact, many years ago, when FreeBSD was my main OS (including on notebook) I went as far as to isolate each app that used internet into its own custom-setup jail [0][1]. I had Firefox, Thunderbird, Pidgin and a few others running in complete isolation from the base system, and from each other. I even had a separate Firefox jail that was only allowed to get out via a Tor socks proxy to avoid leaks (more of an experiment than a necessity, to be fair). Communication between jails was done via commonly mounted nullfs. I have also setup QoS via PF for each of them. They were all running on the host’s Xorg, which was probably also the weakness of this setup. It was a pretty sweet setup, but required quite a bit of effort to maintain, even tho I automated most of the stuff. [...]
The original comment is here: https://news.ycombinator.com/item?id=27709256
Yes, there's always a question of usability, but if you're an advanced user, it's still scary that you're not afforded any control to prevent these incidents unless you go ahead and redesign the whole way all these apps are working all by yourself.
It seems terribly inefficient if every engineer has to do it on their own, and in their own incompatible way. Obviously most people simply give up after a while, since maintaining such a setup might itself be a whole full-time job.
I have looked into doing this many times and it's neither simple nor straightforward.
Specifically: jailing a GUI app that you can interact with on your desktop.
I can't remember what the most promising recipe I saw for this was but it wasn't quite promising enough to compel me to built it up ... and this discussion is always (rightly) hijacked with "just use Qubes" ...
There's also a very interesting read on Qubes-like experience on NixOs with Wayland and XWayland[2,3].
[1]: https://wiki.gentoo.org/wiki/User:Sakaki/Sakaki%27s_EFI_Inst...
[2]: https://roscidus.com/blog/blog/2021/03/07/qubes-lite-with-kv...
I wasn't aware of HP-UX Virtual Vaults, thanks. However, I'd say FreeBSD Jails still had an advantage, due to being free and running on various hardware platforms, and more importantly on commodity hardware.
What the market actually wanted was Docker. And what Docker needed was Linux containers (complicated, flexible, piecewise technology) and not jails, which were higher level abstractions (but yet not high enough) with jargon and framework assumptions that didn't match Docker's needs 1:1.
There are many reasons why FreeBSD jails count not get out for FreeBSD land, one, very important thing is the Linux community's NIH attitude.
Here's an example from my personal name server:
/usr/sbin/jail /jails/www www 10.10.10.36 /lighttpd -f conf/lighttpd.conf
# find /jails/www/usr | wc -l
43
You can also share a lightweight environment with multiple commands - here are two other jail commands:
/usr/sbin/jail /jails/dns ns1 10.10.10.30 /nsd/nsd -c /nsd/nsd.conf
/usr/sbin/jail /jails/dns dns 10.10.10.37 /unbound/unbound -c /unbound/unbound.conf
# find /jails/dns/|wc -l
97
No 'make world' necessary, no building and maintaining of a full FreeBSD system - just the lightest skeleton required for both 'nsd' and 'unbound'.
Packaging an entire system is more about convenience than anything else. It's also pretty difficult to package just the libs one needs when you are dependent on libc and other C libs.
I suspect that if one was really ok with it, some tooling could be built to copy/link in system libs into the rootfs automatically from the host.
It's pretty easy using Nix, e.g. this example defines a container for running a shell script: https://ryantm.github.io/nixpkgs/builders/images/ocitools
That script depends on bash, bash depends on libc, etc. so those dependencies (and only those dependencies) will be put in the container. (See https://nixos.org/guides/nix-pills/enter-environment.html#id... for an example of what dependencies look like in Nix).
> I suspect that if one was really ok with it, some tooling could be built to copy/link in system libs into the rootfs automatically from the host.
Eww, no thanks! I want my containers to be reproducible.
Exodus – relocation of Linux binaries–and all of their deps–without containers - https://github.com/intoli/exodus
I did some reading and it looks like FreeBSD's unionfs or nullfs would handle the file-system part.
Just seems unnecessarily complicated.
Going further, you can place the few executable files you need in a read-only filesystem, so attackers can't copy their own payloads in, further restricting what they can do. You can continue that process with firewall rules that are much more restrictive than what you could use with a more general purpose server, such as blocking any traffic that isn't to or from port 80/443. You can also virtualize the network of a jail, so an attacker wouldn't even get information about the network's layout from a compromised jail.
Typical application attack path:
RCE in the app -> ability to use anything in the OS under the app user privileges -> privilege escalation -> The whole box pwned (including access to all your secrets and management credentials)
Remediation process: wipe the box
Attack path for an app within a jail:
RCE in the app -> ability to use anything in the OS but there is nothing to use, must bring own tools (not always feasible) -> privilege escalation -> JAIL is pwned but not the box
Remediation process: recreate the jail
The tooling is slowly moving in a direction I like, though :)
This is an old post of mine which I happened to find useful. Orchestration of jails moved quite bit forward lately! For example, you can manage your jails quite nicely with containerd today! See great post from Samuel Karp about the topic: https://samuel.karp.dev/blog/2021/05/running-freebsd-jails-w...
apt-cache search jail
firejail - sandbox to restrict the application environment
firejail-profiles - profiles for the firejail application sandbox
firetools - Qt frontend for the Firejail application sandbox
[0]: https://docs.microsoft.com/en-us/virtualization/windowsconta... [1]: https://docs.microsoft.com/en-us/windows-hardware/design/dev...
https://docs.virtuozzo.com/virtuozzo_hybrid_server_7_users_g...
A Docker-like solution with a pretty UI could be really useful for pros. For novices, it could mean a less cumbersome security measure than the restrictions we’ve been experiencing since Catalina.
One thing I like about iocage is how easy it is to grant the jail access to the host ZFS datasets.
On a Jails note, I have had issues creating a jail that can do network inspection. I believe this is an issue with network restrictions of the jails subsystem itself. Eg, I could never run nmap or get mac addresses of remote hosts from within a jail.
Worked well from the limited testing I have done so far