Control Plane

Deep Dive into the Control Plane Resource in CAPT (Cluster API Provider for Tinkerbell)

The Control Plane Resource in the Cluster API Provider for Tinkerbell (CAPT) plays a crucial role in managing the control plane nodes of a Kubernetes cluster. It is typically represented by the KubeadmControlPlane resource within the Cluster API (CAPI) framework. The Control Plane Resource is responsible for provisioning, managing, and maintaining the Kubernetes control plane, ensuring that the API server, etcd, and other critical components are deployed, scaled, and upgraded as needed.

Core Responsibilities of the Control Plane Resource

  1. Control Plane Node Management:
    • The Control Plane Resource defines and manages the lifecycle of control plane nodes in a Kubernetes cluster. This includes initial provisioning, scaling (adding or removing control plane nodes), and upgrading Kubernetes versions.
    • It ensures that the control plane is highly available and resilient by managing multiple replicas of control plane nodes.
  2. Kubeadm Integration:
    • The Control Plane Resource leverages kubeadm for the bootstrap process of control plane nodes. It automates the creation of the control plane by using kubeadm to initialize the Kubernetes API server, etcd, and other essential components.
    • It handles the configuration of kubeadm settings such as the cluster configuration, API server arguments, and etcd configuration.
  3. Infrastructure Integration:
    • The Control Plane Resource integrates with the underlying infrastructure provider (e.g., Tinkerbell) to ensure that the physical or virtual machines provisioned as control plane nodes meet the necessary specifications.
    • It uses an infrastructureTemplate to define the infrastructure configuration for each control plane node, ensuring consistency across all nodes.
  4. High Availability and Upgrades:
    • The Control Plane Resource ensures that the control plane is highly available by managing multiple control plane nodes across different machines.
    • It supports rolling upgrades, allowing for the seamless upgrade of control plane nodes to new Kubernetes versions without disrupting the cluster’s availability.

Core Components of the Control Plane Resource

  1. KubeadmControlPlaneSpec:
    • Role: Defines the desired state of the control plane.
    • Description: The spec section of the KubeadmControlPlane resource outlines the desired configuration for the control plane nodes, including the number of replicas, Kubernetes version, and infrastructure template.
    • Functions:
      • Specifies the number of control plane nodes (replicas) that should be running.
      • Defines the Kubernetes version to be installed on the control plane nodes.
      • Links to the MachineTemplate that defines the infrastructure for control plane nodes.
  2. KubeadmControlPlaneStatus:
    • Role: Represents the current state of the control plane.
    • Description: The status section provides real-time information about the state of the control plane, including the number of ready replicas, updated replicas, and any observed conditions.
    • Functions:
      • Tracks the progress of the control plane in achieving the desired state.
      • Reports on the status of upgrades, including any in-progress rolling updates or scaling operations.
  3. InfrastructureTemplate:
    • Role: Provides the blueprint for creating control plane nodes.
    • Description: The infrastructureTemplate field links to an infrastructure-specific resource (e.g., TinkerbellMachineTemplate) that defines the configuration for each control plane node.
    • Functions:
      • Ensures that all control plane nodes are provisioned with a consistent infrastructure configuration.
      • Facilitates upgrades and scaling by defining the desired configuration for new control plane nodes.
  4. KubeadmConfigSpec:
    • Role: Manages the kubeadm configuration for control plane nodes.
    • Description: The kubeadmConfigSpec defines the settings used by kubeadm to bootstrap the control plane nodes, including the cluster configuration, API server arguments, and etcd configuration.
    • Functions:
      • Controls how the control plane nodes are initialized and joined to the cluster.
      • Allows customization of the kubeadm configuration to meet specific cluster requirements.

Working Example: Using the Control Plane Resource for an Intel NUC-based Kubernetes Cluster

Let’s go through a practical example of defining and managing a Control Plane Resource in CAPT to provision and manage the control plane of a Kubernetes cluster running on Intel NUC hardware.

1. Define the Control Plane Resource

The KubeadmControlPlane resource specifies the desired state of the Kubernetes control plane, including the number of control plane nodes, Kubernetes version, and infrastructure template.

apiVersion: controlplane.cluster.x-k8s.io/v1alpha4
kind: KubeadmControlPlane
metadata:
  name: nuc-control-plane
  namespace: default
spec:
  replicas: 3
  version: v1.21.1
  infrastructureTemplate:
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
    kind: TinkerbellMachineTemplate
    name: nuc-control-plane-template
  kubeadmConfigSpec:
    clusterConfiguration:
      apiServer:
        extraArgs:
          enable-admission-plugins: NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota
    initConfiguration:
      nodeRegistration:
        kubeletExtraArgs:
          cloud-provider: external
    joinConfiguration:
      nodeRegistration:
        kubeletExtraArgs:
          cloud-provider: external

Key components of this configuration:

  • replicas: Specifies the number of control plane nodes (in this case, 3) to ensure high availability.
  • version: Defines the Kubernetes version to be installed on the control plane nodes.
  • infrastructureTemplate: References the TinkerbellMachineTemplate that will be used to provision the control plane nodes on Intel NUC hardware.
  • kubeadmConfigSpec: Specifies the kubeadm configuration for initializing and joining control plane nodes to the cluster.

Apply this resource using kubectl:

kubectl apply -f kubeadm-control-plane.yaml

This command creates the KubeadmControlPlane resource, which will trigger the provisioning and configuration of the control plane nodes.

2. Define the Infrastructure Template

The TinkerbellMachineTemplate resource defines the infrastructure configuration for the control plane nodes managed by the Control Plane Resource.

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
kind: TinkerbellMachineTemplate
metadata:
  name: nuc-control-plane-template
  namespace: default
spec:
  template:
    spec:
      hardwareSelector:
        manufacturer: "Intel"
        plan: "NUC"
      osImage: "ubuntu-20.04"
      userDataSecret:
        name: nuc-control-plane-userdata

Key components of this configuration:

  • hardwareSelector: Specifies the type of hardware (e.g., Intel NUC) to be provisioned as control plane nodes.
  • osImage: Defines the operating system image to install on the hardware.
  • userDataSecret: Reference to a secret that contains the user data for configuring the control plane nodes.

Apply this template:

kubectl apply -f tinkerbell-machine-template.yaml

This template ensures that all control plane nodes are provisioned consistently according to the specified hardware and OS configuration.

3. Scaling the Control Plane

To scale the number of control plane nodes, simply update the replicas field in the KubeadmControlPlane resource:

spec:
  replicas: 5

Apply the updated configuration:

kubectl apply -f kubeadm-control-plane.yaml

The Control Plane Resource will automatically provision the additional control plane nodes, ensuring they match the configuration specified in the infrastructure template.

4. Rolling Updates

To perform a rolling update (e.g., upgrading the Kubernetes version), update the version field in the KubeadmControlPlane resource:

spec:
  version: v1.22.0

Apply the updated configuration:

kubectl apply -f kubeadm-control-plane.yaml

The Control Plane Resource will update the control plane nodes one by one, ensuring minimal disruption to the cluster’s availability.

5. Monitoring the Control Plane

You can monitor the status of the control plane and the individual nodes with the following commands:

kubectl get kubeadmcontrolplanes -A
kubectl get machines -A
kubectl get tinkerbellmachines -A
kubectl get kubeadmconfigs -A

These commands provide insights into the status of the control plane, including the number of ready replicas, any in-progress updates, and the overall health of the control plane nodes.

Conclusion

The Control Plane Resource in CAPT is a critical component for managing the Kubernetes control plane on bare-metal infrastructure like Intel NUCs. By defining the desired state of the control plane, including the number of replicas, Kubernetes version, and infrastructure template, the Control Plane Resource ensures that the control plane is highly available, resilient, and up-to-date. Through its integration with Tinkerbell, the Control Plane Resource abstracts the complexities of hardware provisioning, allowing you to manage Kubernetes control plane nodes with the same ease and flexibility as cloud-based deployments. This approach ensures that your Kubernetes control plane is robust, scalable, and capable of handling production workloads efficiently.