Let's review Docker (opens in new tab)

Feedback and constructive criticism welcome

I would agree that Docker has some flawed fundamentals, that it has many implementation flaws, and that is over-hyped. Out-of-the-box it is even almost useless for real-world production usage.

But - despite agreeing with your words - I wouldn't put them together to reach the same conclusion. The container model with managed images seems to be a big step forwards vs the VM model: it encourages having immutable 'machines' that are disposable, which works very well for cloud architectures.

I would check out the kubernetes project and where it is heading: containers orchestrated across a cluster of machines, with seamless/automatic networking between the containers.

I can only speak for myself. I find docker to be immensely useful. Sure, there were always VMs, but I was very frustrated with multi-vm setups before discovering docker.

At the very least it let me iterate on system configuration / installation procedures more quickly. I have learned a lot just due to tweaking Dockerfiles and rebuilding them. Virtual machines are much slower in this regard.

In Data Science Docker is becoming revolutionary. People tried distributing virtual machines to let others reproduce their work. Dockerfiles are much more reproducible, and don't need a few gigabyte of seperately hosted VM images. Also these VM images usually contain tons of undocumented "state", whereas a Dockerfile is easier to reverse engineer and ultimately very reproducible.

You can include Dockerfiles in all your projects. Other developers can then go in and get started with a minimal amount of guesswork. Turns out "sane" development environments, which probably means one supposedly optimal configuration/setup/framework for all your projects, is the exception rather than then the norm.

The author is viewing/reviewing Docker through a very myopic lens.

I work at a company that leverages Hadoop, and on each developer machine we each have a baby Hadoop cluster (3 nodes) running in VMs at the moment, in addition to a master controller. Hence, at any given time, my machine is actively running 5 OS's (if we include the native OS) and all their associated background tasks. My computer generally idles at 25% CPU usage. Keeps my legs warm in the winter.

We're finally going to dockerize the project and I couldn't be happier. Granted, Mac OSX doesn't natively support the LXC's docker uses (c'mon Apple!), so we'll still be running 1 VM, but that's a huge improvement over our existing implementation. My CPU wouldn't be idling nearly as high, I'm going to recover a ton of RAM, and the boot sequence is going to be substantially faster.

Being that these are developer machines, and simply understanding our use case, we're not worried about malicious tenants breaking out of their environments. It's just not a factor in our situation.

If anything I feel the "article" was a waste of time. I thought I was going to read a great summary on the state of Docker, and instead I got this unexpectedly aggressive piece from developer that couldn't conceive how to fit this particular tool into his toolbelt.

>All of the features which it claims to be helpful are

>either useless or poorly implemented, and it's primary

>benefits can be easily achieved using namespaces directly.

any tutorial ? For me the docker is very easy.

and there are some new alternatives:

- LXD : https://lists.linuxcontainers.org/pipermail/lxc-devel/2014-N...

- Rocket: https://coreos.com/blog/rocket/

- Flockport : http://www.flockport.com/faqs/

- Spoon : https://spoon.net/docs/getting-started/spoon-and-docker

I fundamentally disagree.

Flawed? sure.. useless hype?? yeah well are all the people loving it just stupid.. or is it just placebo?

The author makes a lot of conjecture with very, very little backing.

I love docker. I'm a programmer more than a systems engineer. I've used Linux as my sole computing environment for 6+ years. I've deployed countless LAMP stacks; and countable Haskell/postgres stacks.

For both having an extremely portable development/building environment; and dead-simple disbursement of binaries-with-prereqs, Docker has been INCREDIBLY useful to me.

For actual deployment of single-server apps; it might be a bit more trouble than it's worth, in some cases. I have a couple places where I develop and build in Docker; but actually deploy "raw"; because it is easier/fine.

But when you start considering Coreos clusters and docker containers to utilize them; deployment again is made congitively simpler (to us mere mortals) thinking in terms of containers.

I guess this is click-bait; but even as a passive user of Docker, I find it quite offensive; and not well-grounded.

I opened the article with an open mind; thinking someone smarter than me knew something terrible about docker that was going to bite me in the ass some day.. only to instead get the impression of, either being trolled, or that the author is my favorite type of neck-beard elitist.

If you're renting VMs from a provider like DigitalOcean or Linode, a VM per service can get expensive compared to running multiple containers within a single VM. So, if not Docker, some system for deploying containers in production can still be useful.

While there are some important criticisms in here, there are some misunderstandings of OS-based virtualization with respect to HW-based virtualization in terms of both history and implementation. Certainly, anyone who thinks that "we have reached the point that [hardware virtualized] performance is almost as fast as bare metal" is either cooking their numbers or deliberately limiting the scope of what they are evaluating to purely CPU-bound work; when it comes to anything interacting with the outside world (which is to say, I/O), OS-based virtualization crushes HW-based virtualization.[1] That said, the security criticisms are entirely fair (though they are criticisms of Linux more than of Docker); taken together with the performance win of OS-based virtualization, they form our motivation for Docker containers running as LX-branded zones.[2][3]

[1] http://dtrace.org/blogs/brendan/2013/01/11/virtualization-pe...

[2] http://www.slideshare.net/bcantrill/docker-and-the-future-of...

[3] https://www.joyent.com/developers/videos/docker-and-the-futu...

Some of your points seems valid (although the writing style is hostile), but calling it "unnecessary" is over assumption. You think Docker is only used in deployment, but you are wrong. I use it do grade programming assignments, I distribute Dockerfiles to students, and it is very useful. I believe it is very useful in other aspects as well, just because you are unhappy does not make it unnecessary. I ran almost 30.000 containers on a single machine and it saved me many hours.

I also had a incident with Docker in 0.6, when creating a new container it used to traverse all the containers metadata and when the container count reaches thousands it froze my server and I had to manually reboot it on 3AM, but hey that's okay, it can be fixed, I reported it on Github and it does not happen anymore. It is still useful to me, and still necessary.

"If you expect anything positive from Docker, or its maintainers, then you're shit outta luck.[..]If your development workflow is sane, then you will already understand that Docker is unnecessary[..]Docker would have been a cute idea 8 years ago, but it's pretty much useless today"

Umm ok? Judgement, jumping to conclusions. Take out some of that kind of tone and you have some legitimate criticisms of docker that might be taken more seriously.

There are some real warts in the docker core & community, and a high barrier of entry for multi-node deployments and isolated networks, but on the flipside, deterministic image build & deployment is a big win compared to many other ways of doing this stuff (at scale).

IMO, one of the biggest positives that Docker-like provisioning encourages is clustered-by-default architectures. When you build around failure by assuming nodes come and go (as easily as Docker makes it) your platform availability is likely to be more resilient to some types of failure.

I partially agree with you, except for one very important point: Despite its flaws and hype, it is still incredibly useful.

I just wish it were easier for people to read the contents on the label. Right now everyone thinks it is good for everyone. It isn't, and there are some sharp edges that require RTFM'ing.

I like it for quickly spawning up fresh places to test during development, or for a fresh environment since so many package managers gems, bundler, npm, etc fuck themselves up and install conflicting packages.

So I find this article to be the same as the hype around Docker, useful information surrounded by hyperbole that ruins it?

I don't need a full VM usually, and being just local dev the security isn't an issue, but used Vagrant before Docker for this.

Is there anything else similar?

I came across a few of the issues mentioned in that article while using docker in the last 4 months but I have to say I will never look back.

What I like the most personally is how easy it is to install and experiment with 3d party tools. You want an elastic search stack? In one command you have a webserver hosting Kibana with Elastic Search and Logstash properly configured on your local. Jenkins, Elastic Search, Redis, Postgres, etc: they all have their dockerfiles and can be installed as one liners. Removing them is equally as easy.

Oh and I don't know why it is written that running a Docker registry is "extremely complex". Just like any Dockerized app it is a one-liner.

This new ease-of-install just by itself is worth of my gratitude to the guys that build it.

I always saw Docker as a fast, cheap way to test out deployments. I never considered it for production.

For production I'd use kvm or vmware.

Lastly, you might consider systemd-nspawn. It has gained snapshotting recently (on top of btrfs)

https://plus.google.com/115547683951727699051/posts/W2itNERX...

I find it bewildering that you ran docker in production for 6 months and yet you find that running your own docker registry a "complex" operation. Docker has its flaws, like any other technology, but running a registry took me all of 2 minutes on an EC2 instance. Kinda makes me wonder how much you really understand about a real docker workflow.

I strongly prefer hardware-assisted virtualization wherever possible. I'd like to see more super lightweight OSes like MirageOS, as well as sane trusted computing functionality all the way up the stack, but that might be asking for a bit much.

Look, if you're going to submit your own blog posts to Hacker News --- I assume sleepycal is the same Cal as the Cal who wrote the blog post --- then take the time to polish them up a little, and tone down the flameshow.

Would you like it if I told people your blog post was useless because you buried your thesis in the third-last paragraph and led off with an either astounding or false assertion that you tested docker (which you had previously disliked) by putting it in production for six months? That sort of needs some explanation and story telling. If Docker induced so much bile in you, then how and why did you end up stuck using it in production for six months?

What should the ideal process for building a container look like?

Say I have my python app, how do I transform that into a working container? If a Dockerfile is flawed, whats better?

Ignoring the obviously opinionated cruft and hyper-aggressive uber geek disdain, which appears to make up about 70% of this post, there are still one or two actual statements worth examining. Fwiw I run a small site, fifteen or so instances, and we've been using Docker in our deployment for about a year now.

> Lets say you want to build multiple images of a single repo, for example a second image which contains debugging tools, but both using the same base requirements. Docker does not support this.

Of course it does. It appears that it doesn't support it the way you think it should, but to say that you can't do it is misleading. A base image, and two images that pull from it with the different requirements will solve the problem. You apparently don't like that solution, but that is not the same thing as not having a solution.

> there is no ability to extend a Dockerfile

Yeah, this would be nice. Maybe they will add it. But it is hardly... not even close to... a make-or-break feature. Honestly I think you might just need to refactor your stuff, or perhaps Docker just isn't a fit for what you're doing.

> using sub directories will break build context and prevent you using ADD/COPY

You mean if you include a bunch of stuff in subdirectories that you don't want uploaded to the demon. Again, man, not even close to make or break. You really need to log gigabytes to a subdirectory in your build context? There's _no other way_ you could set that up? We create gigs of logs too, but most of them are events that go to logstash and get indexed into ES. Our file-based logs go to mount points outside the container. We do have images we build using context, where we ADD or COPY multi-gigabyte static data files. Seems to work fine.

> and you cannot use env vars at build time to conditionally change instructions

No, you can't. I'm not sure I would want to. I like the fact that the Dockerfile is a declarative and static description of the dependencies for a deployment. I don't think I want to have to debug conditional evaluation at build time. There are other ways to solve those problems, like refactoring your images.

> Our hacky workaround was to create a base image, two environment specific images and some Makefile automation which involved renaming and sed replacement. There are also some unexpected "features" which lead to env $HOME disappearing, resulting in unhelpful error messages. Absolutely disgusting.

First of all, what exactly is hacky about having a base image and two environment-specific images? I don't know what sort of makefile automation you're talking about, but we do some environment specific sed manipulation of configs at build time, and in some cases at container launch time. Sometimes that makes more sense than having two different versions of the container just to have a very slight change to the config.

Secondly... absolutely disgusting? Is that the sort of language you regularly use in technical writing? Oh, hey, look at the third paragraph: "If you expect anything positive from Docker, or its maintainers, then you're shit outta luck." I guess it is. The strike-out font was a nice touch, man. "I don't really mean this, but you can't help reading it!" Nobody's ever done that before.

> These problems are caused by the poor architectural design of Docker as a whole, enforcing linear instruction execution even in situations where it is entirely inappropriate

You're not talking about linear instruction execution. You're talking about grouping instructions into commited layers. I would much prefer the proposed LAYER command to conditional execution or branching, which is what I assume you mean by non-linear in your comment. But I don't find this to be a serious problem either. That seems to be a pattern with this post: in a year of using Docker to containerize all our services - in-house python code, Django, redis, logstash, elasticsearch, postgresql - I haven't run into these issues that are deal breakers for you. Again, you might want to try to refactor and simplify some of your image builds. It's better to have a few simpler containers talking to each other than to try to cram a complex multi-service deployment into one. But then, I don't know what you're doing, and maybe it's just not suited for containers. You seem to have a strong preference for VMs anyway, so do that.

> However the Docker Hub implementation is flawed for several reasons. Dockerfile does not support multiple FROM instructions (per #3378, #5714 and #5726), meaning you can only inherit from a single image.

This whole post is like a laundry list of Absolutely Critical Things Nobody Ever Needed. I can't imagine a situation in which you'd absolutely have to be able to inherit from multiple images. If you have that situation I would agree it's an indicator Docker won't work the way you currently want to do things. I do agree with you about the occasional speed issues on the hub. But they're giving it to lots of people for free, and to me for a ridiculously low price. If I need better performance I can always run my own registry.

> There are some specific use cases in which containerisation is the correct approach, but unless you can explain precisely why in your use case, then you should probably be using a hypervisor instead.

There are some specific use cases in which virtualization is the correct approach, but unless you can explain precisely why in your use case, then you should probably be using containers instead.

See what I did there?

> If your development workflow is sane, then you will already understand that Docker is unnecessary.

I do like to read even-handed, unbiased reviews of technologies like Docker, even when I already use them. I like to have my world view challenged with an exposition of solid critical points. Maybe someone will write an article like that.

Just wanted to say a huge thank you to everyone who has taken time to reply, the response has been overwhelming! There's still many comments I'm yet to reply to, and will finish replying to these tomorrow evening, but it's now 5am and sleep is required.

I think docker is really successful in term of marketing (no offense), I guess they've learnt a lot from the nodejs/golang's camp :)

Just like few years ago, everyone here on the front page is talking about nosql and seems like everyday, there's a new webscale nosql database being born...but now, I often see more people here inclined toward traditional technologies such as Postgres as it has been greatly improved. (Yes, a lot of aged software are still improving, e.g. MySQL, Apache, PHP or even Perl5)

I am not saying Docker is just a hype, I also don't think it is completely unnecessary, but it is not for me to solve my immediate problems and therefore I would rather review Docker..maybe few years later.

My favourite gotcha for Docker right now is the lack of ssh agent forwarding support with Dockerfile. The only solution appears to be to give the Dockerfile a passwordless build key, which is sorta okay for CI but a total hassle for individual developers.

Who knows better, has the field to himself! says this guy https://www.youtube.com/watch?feature=player_detailpage&v=zO... translation here http://www.rastko.org.rs/knjizevnost/njegos/njegos-mountain_...