undefined | Better HN

story

0 pointschrissnell9y ago0 comments

You're making things really hard on yourself. Boot your nodes with CoreOS and it provides almost everything you need (except Kubernetes itself) out-of-the-box. It all works really well together and you get automatic updates, too. I can't imagine trying to run the cluster we run on Ubuntu, trying to roll my own Docker/etc/flannel installs.

0 comments

lobster_johnson9y ago

I'm sure CoreOS is nice, but we're currently on Ubuntu, and I'm trying to reduce the number of unknown factors and new technologies that we're bringing into the mix. Ubuntu is not the issue here. (FWIW, you don't need Flannel on AWS.)

piva009y ago

Can you expand a bit on why you don't need flannel on AWS? We're currently deploying a k8s cluster and I surely went the flannel route (following the steps of CoreOS guide to k8s) but it'd be nice to remove that setup from our deployment if possible.

lobster_johnson9y ago

AWS has VPCs, allowing you to get a practically unlimited number of private subnets.

In some cloud environments (e.g. DigitalOcean), there's no private subnet shared between hosts, so Kubernetes can't just hand out unique IPs to pods and services. So you need something like Flannel, which can set up a VPC either with UDP encapsulation or VxLAN.

Flannel also has a backend for AWS, but all it does is update the routing table for your VPC. Which can be useful, but can also be accomplished without Flannel. It's also limited to about 50 nodes [1] and only one subnet, as far as I know. I don't see the point of using it myself.

[1] https://github.com/coreos/flannel/issues/164

bboreham9y ago

Could you say how you arrange that the addresses you pick for your pods do not clash with the addresses AWS picks for instances?

1 more reply

lobster_johnson9y ago

Forgot to say: Kubernetes will keep the routing table up to date if you use --allocate-node-cidrs=true. That way, it does exactly the same thing as Flannel with the "aws-vpc" backend.

j / k navigate · click thread line to collapse

0 comments

lobster_johnson9y ago

piva009y ago

lobster_johnson9y ago

AWS has VPCs, allowing you to get a practically unlimited number of private subnets.

[1] https://github.com/coreos/flannel/issues/164

bboreham9y ago

Could you say how you arrange that the addresses you pick for your pods do not clash with the addresses AWS picks for instances?

1 more reply

lobster_johnson9y ago

Forgot to say: Kubernetes will keep the routing table up to date if you use --allocate-node-cidrs=true. That way, it does exactly the same thing as Flannel with the "aws-vpc" backend.

j / k navigate · click thread line to collapse