Kubernetes on-premise setup with kubeadm | Padok

Why would you do this?

• You own your bare metal infrastructure
• You want to deploy Kubernetes to take advantage of it
• You do not wish to migrate your application to the cloud

Why it will not always suit your needs?

• Setup is longer than using cloud providers like GCP or AWS
• Updates to Kubernetes or kubeadm have to be done manually
• Less resilient: if your building has an outage, your service will be down
• Scaling horizontally will require you to buy more servers: Kubernetes cannot auto-scale as much as in a cloud setup

How to do this?

Step 1: The container runtime

First up, you will need to install a runtime for your containers to run in. The most commonly used with Kubernetes is Docker, but you could also use containerd or CRI-O for example. We will go with docker in this tutorial.

If you haven't installed it already, the docker installation is as easy as running the following command on any Linux distro. (Please refer to the docker docs for installation otherwise).

Step 2: Installing kubeadm

What is kubeadm?


kubeadm is a Kubernetes installer. This means it is a tool built for the sole purpose of making it easy to install a Kubernetes cluster on any machine in a matter of minutes. Other installers exist, like kops (repository link) or kubicorn (repository link), but kubeadm is the most commonly used as it is the easiest to use, and is officially supported by all popular cloud providers.

Removing Swap

The first thing to do when installing kubeadm, is to disable swap on all machines (both master nodes and worker nodes). The kubelet, as it functions on the day of writing this article, does not support swap memory. There is a discussion around this feature actually, so if your use case requires the use of swap, check this feature request on GitHub out!

To disable swap, you will need to:

• Run swapoff -a
• Open /etc/fstab file
• Remove any swap entry for the file
  
  

Installing components

Now that we've got that out of the way, we have a couple of things to install on each machine that is part of the cluster. We first need to add the Kubernetes repository to our package manager.

Then, we can install all the components we will need to configure our cluster:

• the kubelet
• kubeadm, which will bootstrap the creation of the cluster
• kubectl as a client to interact with the cluster

Note that we hold the versions for all three of these components. This is key because you do not want to accidentally update your Kubernetes cluster. Or worse, have different nodes running on different versions! Updates can be tricky and have to be planned carefully.

Step 3: Starting the Kubernetes cluster

Launching the cluster is “as easy” as running kubeadm init. However, this single kubeadm command encapsulates a lot of complexity, and there are a few points to watch out for when running it.

• Starting with a simple one, it is good to know that the default CRI Container Runtime Interface is docker. It is possible to change it with the --cri-socket option.
• There are many options for configuring the Pod Network. The default one is Calico, but many others are available. You can pick the one that best suits your needs! Upon setup, kubeadm will add the [node-role.kubernetes.io/master:NoSchedule] taint to the master node.
  This means your master node will never be scheduled to work, and your worker nodes will do all the work. In most cases, this is normal behavior, but if you are setting up a single-node cluster, this will definitely be an issue. You can remove it after initiating kubeadm by using: kubectl taint nodes master key:NoSchedule-.
• By default, kubeadm uses the default network interface of the machine it runs on to set the advertise address for its API server. However, you can choose to use a different one, that matches your setup with the --apiserver-advertise-address=<ip-address> argument.

Once all that is done, you will have an up-and-running master node! The init will take a while, but at the end, kubeadm should provide you with an output looking like this, including instructions on how to set up kubectl to connect to the cluster, and how to have a node join the cluster as a worker:

Step 4: Joining a node to the Kubernetes cluster

To join the Kubernetes cluster as a worker node, you will need to start with step two: Installing kubeadm. Then use the command you got from Step 3: Starting the Kubernetes cluster:

Your node has now joined the cluster, and you can see it when running kubectl gets nodes! There you go, your own bare metal Kubernetes cluster is ready to roll!

If you're looking for tips on how to use Kubernetes, check out our blog, it is filled with them!