Granted I realize the title is "Understanding and Hardening Linux Containers".
However, personally, this just illustrates my frustration and the frustration of others with the Linux world.
They exist in their own little echo chamber. Linux was not the first to create "containers" that are secure. There is no inclusion of other solutions outside Linux.
The reason other solutions do not have these problems is because the authors actually thought about the problem. And if the Linux camp would simply look outside their echo chamber long enough to see how others solved these problems before them this paper might not have been written.
It's just simply frustrating to watch Linux reinvent some wheels, poorly, time after time.
Flame on!
Everybody is just flapping on Linux containers. Meanwhile illumos and SmartOS are eating Linux's lunch with techology which has been working reliably in Swiss banks since 2006, and it is free open source software. What's not to love?
There are T-shirts which say "Go away or I will replace you with a very small shell script." Now it's "Go away or I will virtualize your Linux inside of SmartOS." The damn thing runs like a bandit, while Linux containers keep on flapping trying to find a working solution with papers like these. Why bother, the problem has already been solved, go run SmartOS and enjoy a working solution!
Then I tried Linux (Ubuntu) and everything just worked! It was also very easy to find guides on the web for most things Linux.
They would be eating Linux's lunch if they were used in any wider scale.
A primary objection to Solaris zones and a reason they're not used widely is that they run on Solaris and not on Linux.
Thanks! It was a lot of hard work, I'm glad it's being well received and seems like people are generally finding it useful- the main point of writing it.
I would have liked to talk a lot more about other non-Linux solutions, but that's where most of the NCC Group consulting work has been, which was a main driver for why this info needs to be out there (companies not understanding container security or not "turning it up to 11" in places it needs to be). My background is also mainly Linux, so I stuck to what I know.
Hopefully someone else will release a similar paper on non-Linux solutions/unikernels/hybrid platforms?
There's plenty of room for someone to make a document specific to other platforms for a different audience. There's even room for someone to make a comparison of all platforms. It depends on your goal, your knowledge, and your intended audience.
It's that the paper, while describing the security short comings of the various solutions, illustrates the echo chamber I refer to.
"We have these security issues, that others do not have, but we are not going to look at how they solved these issues because, well, Linux!"
The paper, IMO, would have better served the audience had it included ideas and/or solutions people outside Linux-land developed to solve some of these issues that were written before Linux even attempted a "container".
You are free to disagree.
Out of curiosity, why do you think that Linux got such traction and the BSDs didn't? I keep hearing from the BSD camp that the BSDs are better than Linux at, well, pretty much everything, but there must be some reason why BSDs were not as performant for people than Linux distros.
Out of curiosity, why do you think that
Linux got such traction and the BSDs didn't?
Right around the same time, one Mr. Linus Torvalds was making his own OS.
1 - https://en.wikipedia.org/wiki/UNIX_System_Laboratories,_Inc.....
Complexity at scale: Orchestration frameworks (Rancher, MESOS/Aurora, Docker Swarm, LXD, OpenStack Containers, Kubernetes/Borg, etc) are only recently catching up to the container craze, there are too many competing models to list. Many have questionable or unaudited security or leave major requirements out, such as secret management. While containers may be easy to get working within a workstation or a few servers, scaling them up to production deployment is another challenge altogether, even assuming your application stack can be properly ``containerized''
From my understanding Google was one of the main contributors of the core features that have allowed containers on Linux. Specifically cgroups and LXC. And they have have been running containers for 10 years: https://research.google.com/pubs/pub43438.html
I'm not sure what you consider new, but it's definitely not super new to Google. I'm guessing your issue is that it's fairly new to being used in production across many companies, so security researchers are just starting to work on finding holes in it.
There's a distinction between containers the concept and Linux containers, of course, but yes. You've got it right.
The concept is not new.
This document is more an indictment of the current state of the Linux container ecosystem, rather than containers themselves. Don't conflate the two.
And I do not see how a generic container deployment solution can help with this as distribution of secrets are way too deployment/application specific.
If you want to have a large pool of generic hosts that can run any image, then you've just reduced the size of the pool for some images.
If you choose instead to make every secret available to every host, then you've just spread your individual application secrets across many hosts.
Also, how is secret management a major requirement? That term doesn't even exist outside of open source Silicon Valley trendsters, btw. In the enterprise, only people with regulatory requirements use it, and then it's just called password management or credential management. More important requirements are typically the configuration of the services on the network, basic network security & access control, application security, and the security and training of internal users.
Kubernetes does have a reasonable solution here: http://kubernetes.io/docs/user-guide/secrets/
Honest question: Are illumos zones or FreeBSD jails better designed or is just nobody looking for kernel bugs there? From a usability perspective both win (IMHO) against this Linux mess.
This document is really great through.
Trust me people have been looking for kernel bugs in FreeBSD to exploit jails since it was created. The record their speaks for itself. It's not a lack of eyeballs.
https://docs.docker.com/engine/security/trust/content_trust/
> Content trust is currently only available for users of the public Docker Hub. It is currently not available for the Docker Trusted Registry or for private registries.
> Currently, content trust is disabled by default. You must enable it by setting the DOCKER_CONTENT_TRUST environment variable.
How is the complete lack of image verification until explicitly enabled, and only on public images, a "strong default"?
I'm also mystified by the row for SELinux, where rkt has Optional in scary yellow for some reason, and the other two do not. I suspect that table was the whole point of the independent review and it is fulfilling its purpose handily for Docker. I haven't even read the report and I can identify four suspicious discrepancies in that table alone.
ETA: Compare how the author describes Docker to rkt (it's rkt, not Rkt, too): http://imgur.com/a/D6nEw
I found six inaccuracies in as many minutes after opening the PDF and scrolling to random pages. So I'm not sure that's true. I'm also not the only one making that claim, `spender has too:
https://twitter.com/grsecurity/status/722935691114512385?lan...
I don't want to knock the paper too hard without actually sitting down and reading it (which I'm going to do tonight, and I won't dump raw notes on HN until I've given the paper a serious chance), but I'm not encouraged right off the bat. I've been deep in the rkt integration hole and a lot of stuff referring to rkt is very rough and surface-level and, in five of the cases I mentioned, blatantly factually inaccurate. Same with Docker, too, actually, though the paper is quite obviously partisan (see http://imgur.com/a/D6nEw for example).
But yes, since there's only one entropy pool, attackers can drain /dev/random, causing other programs that rely on /dev/random to block.
All I can say is: on newer kernels, attackers can still drain the pool by using the getrandom syscall, so unless you block that syscall, not mounting /dev/random does not increase the security.
I'd say that's not entirely correct. At all.
I agree with this part. Most containers aren't running as an unprivileged user. Those environments that do support it only support it in a very limited set of os/kernel/whatever versions. Somewhat concerning since containers are getting traction almost everywhere.
Appreciate your efforts. It is good to know that there are people pushing to get this working.
http://regex.info/i/pic/2005-09-06_16:03.28__00002.jpg
Ideas: Cross-linked polymer? Switch to glass? Too heavy. Carbon fiber, epoxy resin composite?
It's weird and wonderful that such technology infrastructure arose and such innovation happens. I'm curious how.
Too bad that Docker and other container solutions defaults to rather broad set of capabilities requiring to use things like
--cap-drop=ALL --cap-add=NET_BIND_SERVICE ...