Docker without Docker (opens in new tab)

Author here - I'm working on a blog post for this and didn't expect this to be posted on HN so soon, but I guess that's what I should expect for posting a link to it in another HN comment! :)

These slides correspond to a workshop I conducted a week ago demonstrating the internals of Docker and how Docker containers can be run without using any of the Docker tools or runtime.

Docker is a great tool, and I'm glad it's gained so much traction. But containerization is still new to many people, and even then there's still a lot of confusion about the difference between Docker and containerization in general. The goal of this presentation isn't to discourage anyone from using Docker, but to outline the lay of the land for people interested in using containers.

Personally, I run containers both with systemd and with Docker. The good news is that it's really easy to switch from one to the other, so there's very little cost to trying it out both ways.

The slides mention that Docker containers have hashes. This is not the case. In fact, it's probably one of the biggest user misunderstandings of Docker. Container IDs and Image IDs are not SHA hashes. Even though it may look like it they are actually just randomly generated 256-bit hex-encoded unique identifiers.

Since the slides also mention that you can use Docker images with a systemd-nspawn/machinectl setup it would be great if they soon supported the v2 Docker Registry and image format which actually does use content-addressable hashes for images.

Use LXC and LXD. It's the best of both worlds. Docker is very limiting and already falling short in features. For instance daemon must run as root which sucks. LXC supports unprivileged namespaces so all users of a system can have their own set of containers. Docker insists on being the parent process, That means when docker dies, so do all your containers.. This is really bad...

You may want to mention that the network and disk isolation are not what someone from Docker would expect by default.

It uses the docker equivalent of "net=host" (which provides better performance at the cost of isolation), and the disk is pointing at a shared "changeroot" on disk, instead of at a layered FS.

Both of these can be better isolated with natted interfaces and a `btrfs` (which has its own reliability issues) layered image, but they are not what you expect by default.

> At its core, your OS is just a bunch of files

"But, you NEED to run the installer on that (bare metal) server!" Nope, I can just boot from knoppix (remember?) and mount the disk and run debootstrap on it.

Every so often I run into programmers and sysadmins that believe these things are a kind of magic. They're not. They're just files on a disk.

Love this presentation. Thank you!

edit: s/mount knoppix/boot from knoppix/

Very interesting. Almost got ready to up my systemd-hate on slide 19: "machinectl -H root@example.com:debian-tree"[1] -- but calmed down and had a look around and saw that it just uses ssh, not some pottering pixiedust kerberos abomination...:

http://www.freedesktop.org/software/systemd/man/machinectl.h...

With proper handling of access (allowing unprivileged users to start containers) along with --bind for the home directory, this could be a viable alternative to Debian's schroot [s].

There's also a complimentary lwn article from 2013 that's worth reading:

https://lwn.net/Articles/572957/

That also contains a quote that explains a bit about systemd (if read maliciously): "As part of the development of systemd, the team looked at various kernel features to see if they were relevant to the project."

At least with this (containers w/log handling etc) we get something for our complexity. Still, having had two seperate machines fail to boot/even come up with a text console with some sensible errors - I'm far from sold on the idea that I want all these features in PID 1.

[1] changed user "foo" to "root" to be a little more clear. Maybe "user1" would work as well - but systemd (unlike lxc etc) requires root?).

[s] https://wiki.debian.org/Schroot

Reminds me that I should probably make write-up of how I set up schroot to allow "source"-access for root, and automagic sessions for a standard user backed by lvm -- the documentation is a bit dense.

tl;dr It's basically a tech demo for systemd, and the systemd-nspawn tool in particular (which you might recall recently gained Docker format support).

This is what normal LXC containers has always done. Systemd nspawn does not yet provide a toolset to wrap these capabilities like the LXC project. Things like userland tools, library of OS templates for containers, networking, features like unprivileged containers that allow non root users to run containers etc.

Lennart Poettering has spoken about containers and btrfs subvolumes and easy snapshots, this could be the direction systemd goes in future for managing the OS with apps in btrfs subvolume containers, with rollback, management etc so this seems like it may mature fairly fast, except unprivileged container support which Lennart does not seem to like.[1]

[1]https://plus.google.com/+LennartPoetteringTheOneAndOnly/post...

As a heavy docker user who only has a couple of years of focused linux experience I found this fascinating, even if you weren't quite ready to post it. Thanks for the slides.

This is almost exactly how we run the Ironic provisioning agent that supports Rackspace OnMetal. We export use Docker to build and export an image, then use CoreOS + systemd-nspawn to run it across every unprovisioned machine.

Anybody else having issues navigating? I'm on Firefox.

that's a good intro to systemd-nspawn and machinectl - which happen to be much nicer to use than docker (yet transparently work with docker images if you want that). its also easier to install since you generally "already have it" and there's no setup.

There's a few things it doesn't do (neither docker, or lxc for that matter) - yet at least - such as mounting fses before container start or manage upgrades.

We're back to chroot jails again?

This slideshow is impossible to read on an iPad; the scrolling isn't synchronized with the width of the screen, and attempting to align a page makes it jump to another. Any solutions?

Edit: Apparently, if you don't touch-move but only tap, you can keep it for, getting out of sync.

Somewhat unrelated. Is there a way to ship a container image with just a "diff" of the default image. Say I build a container with ubuntu 14.04 + some packages installed with apt. I would like to be able to "export" the filesystem, but only the files that are not present in the base os image. The reason I would like this is that I would like to use docker (or systemd-nspawn) to run containers on an device that will not always have access to the internet. It would be nice to be able to add a new package to that device, and not be multi 100 MBs. As long as I can guarantee that the base image is present on the device, I should not need to include it in the image.

So we should also add Docker to the chimera that systemd is becoming?

What is the actual difference here between using debootstrap and LXC?

@chimeracoder:

First thanks, this was really interesting.

Sorry, I must not be that bright, I cannot guess your email from your username :-( (and I don't have a tumbler login, and you haven't enable dm from strangers on twitter), but I wanted to point at that you have a typo on slide 9, s/ journactl/journalctl/.

Combining this with IPFS could be pretty cool!

I haven't used Docker myself. So what value add does Docker provide?

What about the entries in /dev? Is mknod not a thing anymore, or do I not need to have /dev entries in my container root filesystem?

I'm very sad that Hacker School had to change its name (for entirely valid reasons, although it's sad they're there). "Recurse Center" makes it sound like some . . . creationist think-tank.

This is interesting, any recommendations for further reading?

What's old is new. You're describing containers I think.

I feel like HN really loves docking.

So, this is just a systemd commercial pretending to be about docker?