Working with taints and tolerations in Kubernetes

How taints and toleration work?

First, let’s check what exactly taints and toleration in Kubernetes.

Taint

Taints are a property of nodes that push pods away if they don't tolerate this taint. Like Labels, one or more Taints can be applied to a node. This means that the node must not accept any pod that does not tolerate all of these taints.

Toleration

Tolerations are applied to pods, and allow the pods to schedule onto nodes with matching taints.

Taint and toleration

Taints and tolerations work together to ensure that pods are not scheduled onto inappropriate nodes. You can add one or multiple taints and tolerations on your nodes and pods. A taint applied to a node marks that the node should not accept any pods that do not tolerate the taints.

Taint is like key=value:Effect. Taint effect defines the behavior of the pod in regards of the tainted node.

You can assign 3 different values to “effect:

• NoSchedule: if this taint is applied to a node that contains already some pod that doesn’t tolerate this taint, they are not excluded from this node. But no more pods are scheduled on this node if it doesn’t match all the taints of this node. This is a strong constraint.
• PreferNoSchedule: Like the previous one, this taint may not allow pods to be scheduled on the node. But this time, if the pod tolerates one taint, it can be scheduled. This is a soft constraint.
• NoExecute: This taint applies to a node excluding all actual running pods on it and doesn’t allow scheduling if new pods don’t tolerate all taint. This is a strong constraint.

The default value for the operator is “Equal. But it can also be “Exist.

• Equal : key=value:Effect

• Exist : keyExist:Effect

This means that if you change the operator by “exist you don’t have to give a value. If the taint isn’t present on the node, “effect is applied to the pod.

Let's illustrate with schemas. First, when there is no taint, pod can be affected to any node(here Pod1 is attached to Node1, but it could also be on Node2).

<schema-without-taint>

Now, add a taint on the node:
taint=test:NoSchedule


This means if the pod doesn't match this taint, it can’t be scheduled on this node. You can see on the schema below, “the new pod doesn’t have toleration, so it has been affected on Node2. But Pod2, which has toleration that matches the taint, can be affected on Node1.

<schema-with-noschedule-taint>

Now, if we add a taint on the node:
taint=test:NoExecuteIt will exclude all pods which have toleration that doesn’t match the taint.

<schema-with-noexecute-taint>

Deploy node with a taint

To add a taint to an existing node, you can run the following command:

$ kubectl taint nodes node-name key=value:effect


Example: $ kubectl taint nodes node-main taint=test:PrefereNoSchedule

To show taint of your node you can run this command:

$ kubectl describe node

Name: node-main

Roles: <none>

Labels: [...]

Annotations: [...]

CreationTimestamp: Sat, 17 Apr 2021 04:38:19 +0200

Taints: taint=test:PreferNoSchedule

[...]

Deploy a pod with a specific node

Now that your nodes are tainted, here is an example to add tolerations to your pod:

tolerations:

key: "key1"

operator: "Equal"

value: "value1"


Example :

Taint and toleration are useful if you want to work with dedicated nodes. With dedicated nodes, you can create a node pool with very specific parameters(for example high CPU) and use these nodes only for specific applications. You also can separate your applicative pods from those of your secondary applications.

Pay attention to the rules defined in the applied taints. Indeed, if a strong taint is applied to all the nodes and a pod does not have any tolerance, it will not be able to be executed anywhere. Another example: if several taints are applied to a node, the strongest win