- I’ll no longer update this text as I migrated my hosts to Hetzner Online because of constant network issues wit h Scaleway. I’ve created a new blog series about how to setup a Kubernetes cluster at Hetzner Online but since my Ansible playbooks are not provider depended the blog text should work for Scaleway too if you still want to use it. While you can still use PeerVPN to secure the communication between your Kubernetes hosts I switched to WireGuard VPN (see Kubernetes the not so hard way with Ansible - WireGuard for more information).
- Fix link to Kelsey Hightower’s guide
ansible -m setup allshould be run after PeerVPN role was applied
This post is based on Kelsey Hightower’s Kubernetes The Hard Way - Provisioning Compute Resources. Since we don’t use AWS or Google Cloud we don’t have the feature set of this two platforms at hand. But we can work around the shortcoming’s with some tools and one of this tool is PeerVPN.
In part 2 we secured our instances. Since we don’t have networking features like AWS VPC or Google Cloud Engine VPC we create our own secure network with PeerVPN. With PeerVPN you can setup a fully meshed (VPN) network even across datacenter. It’s pretty simple to setup and you can even add your workstation to such a network if you like. You only need at least one host which is publicly available on port 7000 (per default, protocol UDP). That’s important e.g. if you want to run hosts at Scaleway without public IPs (which is perfectly possible). So at least you need one host with public IP. Also if you want to include your workstation which is behind a NAT gateway you need one host with a publicly available IP. PeerVPN will do the rest for you.
I’ve prepared a Ansible role for installing PeerVPN. You can again use
ansible-galaxy install githubixx.peervpn
to install the role or just clone it via git if you like. Basically the PeerVPN settings for all nodes are the same (to be more precise they MUST be the same) - with one exception:
peervpn_conf_ifconfig4. Of course every host will get it’s own IP and that’s the variable we use for this setting. For the PeerVPN IP range I choose
10.3.0.0/24. So for the PeerVPN settings that all hosts share we add a few more variable settings to
group_vars/k8s.yml with the following content (the values are the default settings):
peervpn_conf_networkname: "peervpn" peervpn_conf_psk: "preshared_key_change_it" peervpn_conf_initpeers: "host.example.net 7000" peervpn_conf_enabletunneling: "yes" peervpn_conf_interface: "tap0" peervpn_conf_port: 7000 peervpn_conf_enableipv6: "no"
To create a strong password for the pre-shared key you can create one with
openssl rand -base64 382 | tr -d '\n' && echo
Choose whatever networkname you like for
peervpn_conf_networkname parameter. For
peervpn_conf_initpeers choose a host with a public IP address and check that port 7000 protocol UDP isn’t blocked by your firewall. All hosts will connect to this host for the initial connection. If you secured your instance as described in part 2 of my tutorial you have to add port 7000 (protocol UDP) to
harden_linux_ufw_rules: - rule: "allow" to_port: "7000" protocol: "udp"
For more information about other possible settings have a look at the README of that role but normally you should be fine if you only change the settings I mentioned above.
Finally we need to create a file for every host that is a member of our network. So we create e.g.
host_vars/k8s-controller1.your-domain.tld host_vars/k8s-controller2.your-domain.tld host_vars/k8s-controller3.your-domain.tld host_vars/k8s-worker1.your-domain.tld host_vars/k8s-worker2.your-domain.tld
peervpn_conf_ifconfig4: "10.3.0.xxx/24". Replace
xxx with the number/octet you want to use for that host’s PeerVPN IP address and of course use the real hostname for your hosts for the
hosts_vars file instead of
k8s-XXX.your-domain.tld in my example above. E.g. I usually start at
.101 for controller1 and later with
.111 for worker1.
Include the role into your playbook like in this example:
- hosts: k8s:children roles: - role: githubixx.peervpn tags: role-peervpn
k8s:children is a group which contains other host groups (in our case that’s the group
k8s_worker). Now you can roll out your PeerVPN role via
ansible-playbook --tags=role-peervpn k8s.yml
That limits the tasks which get’s executed and in this case that’s the ones from the PeerVPN role. It takes a while until the hosts will connect via PeerVPN (check
journalctl -f output on the hosts). If you want to apply the role only to one host you can use:
ansible-playbook --tags=role-peervpn --limit=k8s-controller1.your-domain.tld k8s.yml (again of course replace
k8s-controller1.your-domain.tld with the hostname of your first controller).
If you use Ansibles host facts caching as mentioned in part2 make sure that you refresh the cache as we now have a new PeerVPN interface. Use the following command to gather the new host facts:
ansible -m setup all
That’s it for part 3! In part 4 we’ll install a certificate authority which is needed for Kubernetes.