Ask HN: Kubernates on Vsphere

3 pointspaimpozhil9y ago8 comments

We are developing a php/mysql app and it will involve lot of apis and be used by a large group of employees however this is not a public facing internet scale website.

We are using containers (Dockers) so we were looking to setup the system so it is ready with HA/LB/Scale ready from the start.

So we evaluated docker swarm but docker seems to have brought a half cooked swarm mode into mix and its needs it time to become stable/usable.

We wanted to try Kubernates and since our servers are all Vmware ESXI , we gave its Vsphere deployment a go but it didn't work as advertised also looking at their github issues it seems like its maintained by a single person so not sure its the way to go.

We are now looking to evaluate the Baremetal kubernates but running them in the VMs instead but we dont see how to setup the loadbalancer?

Is that even possible?

Also do you believe Kubernates going to support vmware/baremetal and other deployments going forward or they are only going to support the GCE/AWS/DO ?

8 comments

hijinks9y ago

Generally you have to figure out a way to do the load balancing.

Once way is like haproxy in front that has each of the k8 nodes behind it.

How I did it in AWS since I didn't want k8 to create ELBs is

ELB->nginx (on the node outside of k8) -> k8 service nodeport

A nodeport is a type of service that opens the same port on all the k8 nodes to send traffic from the outside to inside the cluster. Even if a node doesn't run the container it can forward traffic to a node that does.

There are also now ingress controllers you can look at that do a lot of the work for you to reload what I did with nginx as soon as you create a service in k8.

paimpozhilOP9y ago

Do you know if this situation of figuring out the LB ourselves is going to change anytime soon? We are happy to wait if you know there is something on the works.

I believe it is possible to wire up service discovery system to add the ips of the backend webservers to the haproxy/nginx but it requires knowing the kubernates internals/api a lot more than now.

However I was looking for a lazy/ just works kind of solution - haha may have to wait or get hands dirty.

hijinks9y ago

no there is no way to do a LB how its handles in AWS/GCE.

You'd have to wire up something yourself to long poll etcd and construct like haproxy/nginx as a LB. Or use a tool like

https://github.com/vulcand/vulcand

1 more reply

nwrk9y ago

Checkout Rancher. From your tone, I guess like Kubernet is complicated for you. Rancher has Kubernet support plus nice GUI and many application templates LAMP stack friendly. Also, load balancer is backed in.

[0] http://rancher.com/ (open source)

paimpozhilOP9y ago

I did indeed give it a trail run on all its modes Rancher/Kubernates/Docker swarm however I didnt feel like its production ready yet but it is certainly interesting.

moondev9y ago

k8s LoadBalancers are designed to be used in the cloud: AWS GCP etc...

As a workaround you can create a service and then tunnel/proxy the service ClusterIP from the master

Kubernetes absolutely supports baremetal. It's one of it's greatest strengths, no lock-in.

paimpozhilOP9y ago

Hey thanks for the response, will look into it.

j / k navigate · click thread line to collapse

Ask HN: Kubernates on Vsphere

3 pointspaimpozhil9y ago8 comments

We are developing a php/mysql app and it will involve lot of apis and be used by a large group of employees however this is not a public facing internet scale website.

We are using containers (Dockers) so we were looking to setup the system so it is ready with HA/LB/Scale ready from the start.

So we evaluated docker swarm but docker seems to have brought a half cooked swarm mode into mix and its needs it time to become stable/usable.

We are now looking to evaluate the Baremetal kubernates but running them in the VMs instead but we dont see how to setup the loadbalancer?

Is that even possible?

Also do you believe Kubernates going to support vmware/baremetal and other deployments going forward or they are only going to support the GCE/AWS/DO ?

8 comments

hijinks9y ago

Generally you have to figure out a way to do the load balancing.

Once way is like haproxy in front that has each of the k8 nodes behind it.

How I did it in AWS since I didn't want k8 to create ELBs is

ELB->nginx (on the node outside of k8) -> k8 service nodeport

There are also now ingress controllers you can look at that do a lot of the work for you to reload what I did with nginx as soon as you create a service in k8.

paimpozhilOP9y ago

Do you know if this situation of figuring out the LB ourselves is going to change anytime soon? We are happy to wait if you know there is something on the works.

I believe it is possible to wire up service discovery system to add the ips of the backend webservers to the haproxy/nginx but it requires knowing the kubernates internals/api a lot more than now.

However I was looking for a lazy/ just works kind of solution - haha may have to wait or get hands dirty.

hijinks9y ago

no there is no way to do a LB how its handles in AWS/GCE.

You'd have to wire up something yourself to long poll etcd and construct like haproxy/nginx as a LB. Or use a tool like

https://github.com/vulcand/vulcand

1 more reply

nwrk9y ago

[0] http://rancher.com/ (open source)

paimpozhilOP9y ago

I did indeed give it a trail run on all its modes Rancher/Kubernates/Docker swarm however I didnt feel like its production ready yet but it is certainly interesting.

moondev9y ago

k8s LoadBalancers are designed to be used in the cloud: AWS GCP etc...

As a workaround you can create a service and then tunnel/proxy the service ClusterIP from the master

Kubernetes absolutely supports baremetal. It's one of it's greatest strengths, no lock-in.

paimpozhilOP9y ago

Hey thanks for the response, will look into it.

j / k navigate · click thread line to collapse