I’ve had all sort of difficulties installing Docker. By hand it’s not trivial to get a secure install. Docker machine is great except it’s often broken. The Docker machine dev team is a tired, understaffed bunch that’s always playing a sisyphean whack-a-mole against dozens of cloud providers and very needy posters on Github, myself included.
Kubernetes on the other hand is trivial with GKE. It’s great for single node deployments. I run a single node on GKE and it’s awesome, easy, and very cheap. You can even run preemptible instances. The myth that kubernetes is complicated is largely perpetuated by the same kind of people who say React is complicated: the people who’ve not tried it.
And like React, once you try kubernetes you never go back. Kubernetes is actually the orchestration equivalent of React. You declare what should be true, and Kubernetes takes care of the rest.
And the features it provides are useful for any-sized application! If you try kubernetes you quickly discover persistent volumes and statefulsets, which take away most of the complexities out of stateful applications (ie most applications). You also discover ingress resources and controllers, which make trivial so many things that are difficult with Swarm, like TLS termination. Swam doesn’t have such features, which any non-trivial app (say, Django, wordpress, etc) benefits from tremendously.
How do I install GKE on my servers? ;)
> By hand it’s not trivial to get a secure install.
The default install (basically, adding a repo and apt-get install docker-ce on Debian and derivatives - trivial to automate with Ansible) is reasonably secure if you view Docker as a tool for packaging and task scheduling with some nice extras and don't buy the marketed isolation properties. It only listens for commands on a local socket, and permissions are sane. I haven't looked into Swarm mode protocol traffic, though but I don't think it's tweakable anyway.
> The myth that kubernetes is complicated is largely perpetuated by the same kind of people who say React is complicated: the people who’ve not tried it.
I've tried K8s. I've set up a test cluster, it worked, I wrote some YAML, it worked, all good. So I've worsened the conditions (explicitly going into "I want things to break" territory) and made it fail. I've researched how hard it is to diagnose the problem and fix it - it happened to be complicated. At least, for me. Just felt that "if something goes wrong here, I'll have bad time trying to fix it". Surely, this is not the case on GKE where you don't run and don't manage the cluster.
I had somewhat similar experience with Docker and Docker Swarm mode, and it was significantly easier to dive into the code, find out the relevant parts and see what's going on.
> difficult with Swarm, like TLS termination
YMMV, but I just add some labels to the service and Traefik does the rest. ;)
(But, yeah, Traefik with Swarm requires some reasonable but not exactly obvious networking logic. May take one deployment of "why I'm getting 504s?!" to figure it out. And Traefik needs access to manager nodes to work with built-in Swarm service discovery.)
Great question! All major Cloud providers offer managed Kubernetes services:
https://cloud.google.com/kubernetes-engine
https://azure.microsoft.com/en-us/services/container-service
https://www.ibm.com/cloud/container-service
The choice is Cloud + Kubernetes vs. roll everything on your own hardware.
Running your own hardware is a major IT effort. Kubernetes is just a part of that effort. Then you have take care about planning, provisioning, logging, monitoring, alerting, auditing, networking, storage, oncall.
You either run Cloud + Kubernetes and get rolling in 15 minutes, or you hire IT headcount.
I agree with what you say. I'm not tying to say people should all jump to k8s. Having options on the market is great.
But I was trying to refute the notion that Kubernetes has no advantages unless you're running a huge cluster. My main points where:
* It works great with 1 node.
* It comes with many features that Swarm does not have that are useful even at 1 node (PersistentVolumes, StatefulSets are biggest for me, though there are _many_ more I wouldn't want to go without anymore).
* Docker is not trivial to set up, either.
> How do I install GKE on my servers? ;)
Yes, of course. I was just saying there's a solid option to start exploring quickly.
> It only listens for commands on a local socket.
This is kind of a non-starter, isn't it? Of course it's easy to apt-get install docker, but then you want to control it remotely, right? Once you realize how nice it is to control Docker remotely, it's hard to imagine life before.
Please get some experience with that, then re-evaluate whether Docker or Kubernetes is easier for small deployments.
But with your easier Swarm setup, how do you then attach cloud disks directly to your docker container, like a PersistentVolume affords? That one feature makes basically anything worth it, IMO. Most apps are stateful.
Yes, true, but that's apple vs oranges. Swarm doesn't compete against GKE, it competes against Kubeadm. Because sometimes one actually can't or won't use cloud services.
I don't understand ur comment, yea using something in preinstalled in the cloud is easier than installing something on your own.
I regularly use a script like this, and it's never failed me unless it runs out of disk space: https://github.com/coventry/InformationDropout/blob/master/p...
I haven't used but I have used DC/OS, Mesos, and Marathon extensively, which is not a setup I'd do for a small number of nodes personally.
But I stress it's not about the scaling. It's about the features even at a single node. I wouldn't be spending time writing this stuff had it not been revelatory for me.
PS. If you're not able to try it on GKE, there's also https://github.com/kubernetes/kops. GKE is great for trying, at least, though. Just get a cluster running and get rolling with the concepts. Then you'll know what it's all about. There isn't that much to it, and what there is to it is great and well documented with a great community.
I would like to note that DC/OS now has Kubernetes [Beta]. You can manage Kubernetes resources on your DC/OS cluster now.
Here are the instructions on Github to get started: https://github.com/mesosphere/dcos-kubernetes-quickstart You can also join the Slack to help you on you out: http://chat.dcos.io/ (the #kubernetes channel)
Would you use DC/OS-marathon (vs k8s) now if you were to make that decision now.
I've heard that mesos stack is good for machine learning/big data stacks but how does marathon compare for deploying webapps.
https://kubernetes.io/docs/tutorials/
(Disclosure: I run CNCF, which hosts Kubernetes and funded the EdX course.)
As for bug reports - zero feedback from anyone on that thread from maintainers. If you are one of the maintainers - it might be good to write this comment on that thread instead of HN
* - I understand that web developers might be a small percentage of users and my case doesn't represent everybody
If you want to help send in diagnostic reports for the Docker guys.
Find out what is causing the CPU spikes with
docker stats
screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
Docker for Mac/Windows is a great product - it has allowed me to roll out Docker to developers who wouldn't otherwise deal with Vagrant or other solutions
Right now, I am running 10 containers on my Mac, and my highest CPU in Intellej. Docker is running consistently with about 5-10% of the CPU.
I just installed Docker edge. We'll see how it goes in the long run but so far with k8s enabled hyperkit uses 25% of a virtual core.
Anyways, I envision this being very useful for development, may even replace my docker-compose based test setup.
Many people ask why would someone use an orchestrator on a small cluster (dozens of hosts). Why not? Swarm is very easy to manage and maintain, using Puppet or Ansible is not less complicated at all.
The future of Docker, Inc. is of course Docker EE, and the future of Docker EE is _not_ the scheduler, it's everything around it.
The idea that dockerized software somehow is less dependent on configuration management seems to be a popular and completely misguided one. The two trends are completely separate, but I would argue from experience that unless you have absolutely nailed the configuration and integration of all your moving parts, don't even look at containers yet.
Containers tend to lead to more moving parts, not less. And unless you know how to configure them, and perhaps even more importantly how to test them, that will only make matters worse.
I’d love to know who these mythical folks are?
1) dozens of hosts (heck, hosts >1) is exactly why you need orchestration
2) while there are huge deployments across the globe, I wouldn’t consider “dozens of hosts” small by no means. That’s actually probably above average.
3) k8s is actually easier to maintain than you allude. I see these comments about Swarm over k8s generally from folks who never even tried it (or did so years ago), is that the case here?
It just sounds accurate to say it is like trying to be like Google. Well if it is but less complicated then there is no down side.
The complexity overhead of kubernetes may be premature when you don't have massive requirements
Swarm was always much simpler to get started with, so some people will prefer it. But it's clear to me which one won.
Because I already have Docker for Mac installed to be able to build and test images I think it's useful to have this local k8s integration.
Given that running docker on a Mac has been a long and bumpy journey, I’m not sure I’d want to bundle the two together.
What am I missing?
If Docker's local Kubernetes install provides a way to connect to custom registries (e.g. GCR) without installing a third party plugin onto every minikube, I'd consider that a major win.
https://github.com/kubernetes/minikube/blob/master/docs/inse...
If you want a standalone installation of Kubernetes on a Mac try minikube.
https://twitter.com/alexellisuk/status/949595379326210048
Make sure you do a context-switch for kubectl too.
I see some people talking about Swarm vs Kubernetes. Swarm has always maintained a less modular approach which made it simpler to work with - it's obviously not going anywhere since there are customers relying on it in production.
Out of the two it's the only one that can actually fit into and run on a Raspberry Pi Zero because K8s has such high base requirements at idle in comparison. For a better comparison see my blog post - https://blog.alexellis.io/you-need-to-know-kubernetes-and-sw...
While the two solutions are obviously with different goals in mind, one being to fully run the kubernetes setup locally, and the other one to run a few docker containers who talk to one another, if it's for the purpose of running a simple-ish dev environment, in my experience, docker-compose is much faster and simpler than minikube.
I really wish I could remember what SPOF was pertaining to, but I just can't remember. Does anyone have any idea if this is still relevant/accurate information?
He told me this maybe 2-3 years ago, so I was wondering how things have changed since then, or if anyone knows what he might have been talking about.
Note that even if you don't have HA, Kubernetes being a SPOF isn't necessarily critical. Barring some kind of catastropic, cascading fault that affects multiple nodes and requires rescheduling pods to new nodes, a master going down doesn't actually affect what's currently running. Autoscaling and cronjobs won't work, clients using the API will fail, and failed pods won't be replicated, but if the cluster is otherwise fine, pods will just continue running as before. Ny analogy, it's a bit like turning off the engines during spaceflight. You will continue to coast at the same speed, but you can't change course.
Docker is primarily for running Linux applications on Linux (yes, I know there are things like Joyent SDC, Docker Engine on Windows etc).