Hetzner DIY Private Networking with Tinc (opens in new tab)

(romantomjak.com)

87 pointsromantomjak7y ago37 comments

37 comments

32 comments · 5 top-level

pstadler7y ago· 13 in thread

Use WireGuard[1] instead. It's way faster than Tinc and other userland VPN implementations. I've been using it for the same purpose as the author of the article and it has been rock solid - not a single issue during almost two years. Setup and configuration is a breeze[2].

[1] https://www.wireguard.com/ [2] https://github.com/hobby-kube/guide#wireguard-setup

Edit: Benchmarks on Hetzner Cloud (1vCPU, 2GB)

  $ iperf3 -c kube1
  Connecting to host kube1, port 5201
  [  4] local 10.0.1.2 port 57622 connected to 10.0.1.1 port 5201
  [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
  [  4]   0.00-1.00   sec  77.2 MBytes   647 Mbits/sec   79   1.37 MBytes
  [  4]   1.00-2.00   sec  78.8 MBytes   661 Mbits/sec    0   1.51 MBytes
  [  4]   2.00-3.00   sec  81.2 MBytes   681 Mbits/sec    0   1.62 MBytes
  [  4]   3.00-4.00   sec  85.0 MBytes   713 Mbits/sec  134   1.20 MBytes
  [  4]   4.00-5.00   sec  80.0 MBytes   671 Mbits/sec    0   1.28 MBytes
  [  4]   5.00-6.00   sec  77.5 MBytes   651 Mbits/sec    0   1.33 MBytes
  [  4]   6.00-7.00   sec  88.8 MBytes   745 Mbits/sec    0   1.37 MBytes
  [  4]   7.00-8.00   sec  73.8 MBytes   619 Mbits/sec    0   1.39 MBytes
  [  4]   8.00-9.00   sec  78.8 MBytes   661 Mbits/sec    0   1.41 MBytes
  [  4]   9.00-10.00  sec  80.0 MBytes   671 Mbits/sec    0   1.42 MBytes

gant7y ago

Running Kube on their cloud servers? Well have fun with that, the "vCore" is a very inconsistent unit unless you get their dedicated core servers. I moved back to Hetzner Bare Metal because you can't have anything that will push the resource boundaries on these boxes.

Also regarding Wireguard, I really like how tinc will find a new path and allows you to route over other nodes as needed. Wireguard can't really do that out of the box, every link is 1:1. You can of course setup something on top of that, but I miss the ease with which tinc does this.

chrismeller7y ago

I was actually surprised at the lackluster performance on the cloud products as well and recently spun up a dedicated box for a workload that actually required consistent performance. I never expected the performance to match a bare metal option of course, but coming from any of the other cloud providers I expected it to be more equivalent than it turned out to be.

subway7y ago

along the lines of automatically re-routing, tinc also has some neat anycast-like capabilities -- you can assign the same ip to multiple nodes, and the lowest latency/shortest route node wins

jmngomes7y ago

I was considering using autossh to create a private link between servers, because in my case it's only a handful of servers.

Can you comment on how stable a Wireguard tunnel is? Did you manage to get the link/VPN to stay up permanently with little to no maintenance?

amaccuish7y ago

There isn't really a up/down of wireguard, once the interfaces are configured, you just start pumping packets through, it's pretty invisible.

pstadler7y ago

Found it to be incredibly stable, plus the links are self-healing due to its design.

jamescun7y ago

I second Wireguard, I've been using it recently instead of an overlay network in Kubernetes (configured as a kubenet). Incredibly easy to set up and very performant.

amq7y ago

Could you describe how you did it?

cbluth7y ago

Some information on how to do this would be awesome

zaarn7y ago

I would use WireGuard tbh, but I use pfSense for Networking and there doesn't seem to be a userspace implementation available that runs on it (I did try some FreeBSD binary that I copied over but that didn't quite work out).

moviuro7y ago

The -go version should work, though (if compiled correctly). https://git.zx2c4.com/wireguard-go/

ibotty7y ago

I hope https://github.com/gsliepen/tinc/issues/179 becomes a reality: tinc ui and features on top of wireguard!

romantomjakOP7y ago

I did know about this, but it looks very interesting! Will defo check it out, thanks!

TomMarius7y ago· 5 in thread

Isn't the point of DO's private networking that you don't need to encrypt the traffic? Or is it just internal, but not private?

jarym7y ago

Well private just means it’s isolated from other networks - it doesn’t mean that your ‘private’ network can’t be snooped (by Hetzner, hackers, etc.)

We’re experimenting with Wireguard on all internal hosts and disabling SSL.

chrismeller7y ago

Yeah, I found that an odd comparison to make as well. If you want encrypted traffic that's all well and good, but there's no reason to assume that the private network is going to be any different performance wise than the exact same encrypted solution over the public interface - a network is a network is a network in this case.

Since the goal was to have a private network between your own boxes, the encryption was only really "required" to protect private data because it had to transit the public network in Hetzner. Since DO provides a private network natively there's (in theory) no justification for the encryption, which means you'd get native performance, hence the advantage.

pstadler7y ago

Are you sure DO's private network traffic is actually encrypted or even isolated? Back some time ago, any host within the same private network could be reached. I wasn't surprised to see connection attempts from random hosts on eth1.

1 more reply

nsomaru7y ago

I’ve heard DOs internal traffic is internal not private. You’ve got to lock your boxes down anyways.

nicolaslem7y ago

This changed a few months ago. You still share the same private network with everyone else in the DC, but only machines on your account can communicate with each other.

1 more reply

_Codemonkeyism7y ago· 5 in thread

What about Zerotier with Hetzner?

jbverschoor7y ago

Yeah I'm not sure why zerotier is not getting enough credits here on HN. It works flawlessly, is super fast, easy, works on iphone, and they have a small hardware box now.

chrisper7y ago

Zerotier doesn't do PFS Perfect Forward Secrecy... and somehow it is too easy to add new clients to the network without you noticing.

radiowave7y ago

IIRC in Zerotier new clients are easily added, but traffic to and from them is blocked by default, until you approve them in the web interface.

manigandham7y ago

New clients have to be approved before they can join. How would you not notice?

1 more reply

jbverschoor7y ago

It's too bad you mae these comments without actually having tried it.

mwest7y ago· 3 in thread

You can achieve something similar with Hetzner's recently introduced "vSwitch feature". Works across their different DCs, which is nice. Some docs here: https://wiki.hetzner.de/index.php/Vswitch/en

I've been using ZeroTier to give a common backplane to my Hetzner servers, DO droplets and AWS instances.

jmngomes7y ago

I understand this may not be an issue in your case, but vSwitches won't encrypt your data in transit between servers, unlike a VPN or ssh tunnel.

gant7y ago

It depends. I've seen some shit on cheap bare metal providers, including getting ARP poisoned on Online.net.

Hetzner has been great overall. They've been very very helpful in documenting me reacting to abuse emails too when I got into some user-generated-content related legal trouble.

1 more reply

chrismeller7y ago

vSwitch is only, AFAIK, available on their dedicated servers (well, anything in Robot... which also includes their legacy virtualized product). OP is using their new cloud offering, which doesn't have an equivalent option.

danielh7y ago· 1 in thread

> Normally you only get one public IP and no private interfaces.

From my understanding, this statement is not quite correct, as Hetzner allows you to set up VLANs:

> With the vSwitch feature, you can connect your dedicated root servers in multiple locations to each other using VLAN via the administration interface Robot.

You probably still want to encrypt the traffic passing through those VLANs.

They also offer the option to install custom hardware, so you might even be able to get a second NIC connected to your own private switch.

chrismeller7y ago

That only applies to their dedicated servers. OP is using their cloud offering, which doesn't support this feature or custom hardware.

j / k navigate · click thread line to collapse

37 comments

32 comments · 5 top-level

pstadler7y ago· 13 in thread

[1] https://www.wireguard.com/ [2] https://github.com/hobby-kube/guide#wireguard-setup

Edit: Benchmarks on Hetzner Cloud (1vCPU, 2GB)

  $ iperf3 -c kube1
  Connecting to host kube1, port 5201
  [  4] local 10.0.1.2 port 57622 connected to 10.0.1.1 port 5201
  [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
  [  4]   0.00-1.00   sec  77.2 MBytes   647 Mbits/sec   79   1.37 MBytes
  [  4]   1.00-2.00   sec  78.8 MBytes   661 Mbits/sec    0   1.51 MBytes
  [  4]   2.00-3.00   sec  81.2 MBytes   681 Mbits/sec    0   1.62 MBytes
  [  4]   3.00-4.00   sec  85.0 MBytes   713 Mbits/sec  134   1.20 MBytes
  [  4]   4.00-5.00   sec  80.0 MBytes   671 Mbits/sec    0   1.28 MBytes
  [  4]   5.00-6.00   sec  77.5 MBytes   651 Mbits/sec    0   1.33 MBytes
  [  4]   6.00-7.00   sec  88.8 MBytes   745 Mbits/sec    0   1.37 MBytes
  [  4]   7.00-8.00   sec  73.8 MBytes   619 Mbits/sec    0   1.39 MBytes
  [  4]   8.00-9.00   sec  78.8 MBytes   661 Mbits/sec    0   1.41 MBytes
  [  4]   9.00-10.00  sec  80.0 MBytes   671 Mbits/sec    0   1.42 MBytes

gant7y ago

chrismeller7y ago

subway7y ago

along the lines of automatically re-routing, tinc also has some neat anycast-like capabilities -- you can assign the same ip to multiple nodes, and the lowest latency/shortest route node wins

jmngomes7y ago

I was considering using autossh to create a private link between servers, because in my case it's only a handful of servers.

Can you comment on how stable a Wireguard tunnel is? Did you manage to get the link/VPN to stay up permanently with little to no maintenance?

amaccuish7y ago

There isn't really a up/down of wireguard, once the interfaces are configured, you just start pumping packets through, it's pretty invisible.

pstadler7y ago

Found it to be incredibly stable, plus the links are self-healing due to its design.

jamescun7y ago

I second Wireguard, I've been using it recently instead of an overlay network in Kubernetes (configured as a kubenet). Incredibly easy to set up and very performant.

amq7y ago

Could you describe how you did it?

cbluth7y ago

Some information on how to do this would be awesome

zaarn7y ago

moviuro7y ago

The -go version should work, though (if compiled correctly). https://git.zx2c4.com/wireguard-go/

ibotty7y ago

I hope https://github.com/gsliepen/tinc/issues/179 becomes a reality: tinc ui and features on top of wireguard!

romantomjakOP7y ago

I did know about this, but it looks very interesting! Will defo check it out, thanks!

TomMarius7y ago· 5 in thread

Isn't the point of DO's private networking that you don't need to encrypt the traffic? Or is it just internal, but not private?

jarym7y ago

Well private just means it’s isolated from other networks - it doesn’t mean that your ‘private’ network can’t be snooped (by Hetzner, hackers, etc.)

We’re experimenting with Wireguard on all internal hosts and disabling SSL.

chrismeller7y ago

pstadler7y ago

1 more reply

nsomaru7y ago

I’ve heard DOs internal traffic is internal not private. You’ve got to lock your boxes down anyways.

nicolaslem7y ago

This changed a few months ago. You still share the same private network with everyone else in the DC, but only machines on your account can communicate with each other.

1 more reply

_Codemonkeyism7y ago· 5 in thread

What about Zerotier with Hetzner?

jbverschoor7y ago

Yeah I'm not sure why zerotier is not getting enough credits here on HN. It works flawlessly, is super fast, easy, works on iphone, and they have a small hardware box now.

chrisper7y ago

Zerotier doesn't do PFS Perfect Forward Secrecy... and somehow it is too easy to add new clients to the network without you noticing.

radiowave7y ago

IIRC in Zerotier new clients are easily added, but traffic to and from them is blocked by default, until you approve them in the web interface.

manigandham7y ago

New clients have to be approved before they can join. How would you not notice?

1 more reply

jbverschoor7y ago

It's too bad you mae these comments without actually having tried it.

mwest7y ago· 3 in thread

You can achieve something similar with Hetzner's recently introduced "vSwitch feature". Works across their different DCs, which is nice. Some docs here: https://wiki.hetzner.de/index.php/Vswitch/en

I've been using ZeroTier to give a common backplane to my Hetzner servers, DO droplets and AWS instances.

jmngomes7y ago

I understand this may not be an issue in your case, but vSwitches won't encrypt your data in transit between servers, unlike a VPN or ssh tunnel.

gant7y ago

It depends. I've seen some shit on cheap bare metal providers, including getting ARP poisoned on Online.net.

Hetzner has been great overall. They've been very very helpful in documenting me reacting to abuse emails too when I got into some user-generated-content related legal trouble.

1 more reply

chrismeller7y ago

danielh7y ago· 1 in thread

> Normally you only get one public IP and no private interfaces.

From my understanding, this statement is not quite correct, as Hetzner allows you to set up VLANs:

> With the vSwitch feature, you can connect your dedicated root servers in multiple locations to each other using VLAN via the administration interface Robot.

You probably still want to encrypt the traffic passing through those VLANs.

They also offer the option to install custom hardware, so you might even be able to get a second NIC connected to your own private switch.

chrismeller7y ago

That only applies to their dedicated servers. OP is using their cloud offering, which doesn't support this feature or custom hardware.

j / k navigate · click thread line to collapse