Infrastructure Provider

Deep Dive into the Infrastructure Provider in CAPT (Cluster API Provider for Tinkerbell)

The Infrastructure Provider in the Cluster API Provider for Tinkerbell (CAPT) is a critical component that abstracts the underlying bare-metal infrastructure managed by Tinkerbell. It enables the Cluster API (CAPI) to interact with Tinkerbell, allowing Kubernetes clusters to be provisioned, scaled, and managed on bare-metal servers.

Core Responsibilities of the Infrastructure Provider

  1. Resource Abstraction:
    • The Infrastructure Provider abstracts the details of the underlying hardware (e.g., Intel NUCs, servers) and exposes them in a way that the Cluster API can interact with. This abstraction layer allows CAPI to manage clusters in a cloud-agnostic way, while the Infrastructure Provider handles the specifics of the bare-metal provisioning.
  2. Tinkerbell Integration:
    • The Infrastructure Provider is responsible for integrating with Tinkerbell’s API, converting the Kubernetes cluster definitions and machine specifications into Tinkerbell workflows that provision the physical hardware.
    • It ensures that the appropriate workflows are executed on Tinkerbell to bring the physical servers up to the desired state, including tasks like network booting, OS installation, and initial configuration.
  3. Machine Lifecycle Management:
    • The Infrastructure Provider manages the full lifecycle of the machines in the cluster. This includes creating new machines, updating existing machines (e.g., during an upgrade), and deleting machines when they are no longer needed.
    • It handles the process of associating a Kubernetes Machine resource with a physical server provisioned by Tinkerbell.
  4. State Reconciliation:
    • Similar to other components in CAPI, the Infrastructure Provider constantly monitors the desired state of cluster resources and reconciles them with the actual state. If a discrepancy is detected (e.g., a machine goes down or is missing), the Infrastructure Provider triggers the necessary actions to restore the desired state.

Core Components of the Infrastructure Provider

  1. TinkerbellCluster Resource:
    • Role: Represents the infrastructure configuration for the entire Kubernetes cluster.
    • Description: This resource defines how the Kubernetes cluster should interact with the Tinkerbell environment, including details like control plane endpoints, network settings, and any Tinkerbell-specific configurations.
  2. TinkerbellMachine Resource:
    • Role: Represents an individual bare-metal machine in the Kubernetes cluster.
    • Description: This resource defines the specifics of a machine that should be provisioned by Tinkerbell, including hardware type, operating system, and network settings. The Infrastructure Provider uses this resource to map a Kubernetes Machine resource to a physical server.
  3. TinkerbellMachineTemplate Resource:
    • Role: A reusable template for creating TinkerbellMachine resources.
    • Description: This resource defines a blueprint for machines that can be used across multiple clusters or machine deployments. It includes the specifications for the machines, such as the hardware profile, OS image, and other provisioning details.
  4. TinkerbellClusterController:
    • Role: Manages the lifecycle of the TinkerbellCluster resource.
    • Description: This controller is responsible for creating, updating, and deleting the TinkerbellCluster resource. It ensures that the cluster is correctly configured to use Tinkerbell as its infrastructure provider.
  5. TinkerbellMachineController:
    • Role: Manages the lifecycle of individual machines.
    • Description: This controller handles the provisioning, scaling, and deletion of machines within the cluster. It communicates with Tinkerbell to ensure that the physical hardware is provisioned according to the specifications in the TinkerbellMachine resource.

Working Example: Using the Infrastructure Provider for Intel NUC Hardware

Let’s walk through an example of how the Infrastructure Provider in CAPT works to provision and manage a Kubernetes cluster on Intel NUC hardware.

1. Define the TinkerbellCluster Resource

The TinkerbellCluster resource specifies the cluster-wide infrastructure settings, such as the control plane endpoint and network configurations.

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
kind: TinkerbellCluster
metadata:
  name: my-cluster-infra
  namespace: default
spec:
  controlPlaneEndpoint:
    host: "192.168.1.100"  # IP address where the control plane API server will be accessible
    port: 6443

Apply the resource:

kubectl apply -f tinkerbell-cluster.yaml

This resource tells the Infrastructure Provider how to configure the overall cluster infrastructure in the Tinkerbell environment.

2. Define the TinkerbellMachineTemplate Resource

Next, define the TinkerbellMachineTemplate resource, which will serve as a blueprint for the machines that will be provisioned by Tinkerbell:

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
kind: TinkerbellMachineTemplate
metadata:
  name: my-cluster-control-plane-template
  namespace: default
spec:
  template:
    spec:
      hardwareSelector:
        manufacturer: "Intel"
        plan: "NUC"
      osImage: "ubuntu-20.04"  # Specify the OS image to be installed
      userDataSecret:
        name: my-cluster-userdata
      providerID: "tinkerbell://my-cluster-control-plane"

Apply this template:

kubectl apply -f tinkerbell-machine-template.yaml

This template will be used by the Infrastructure Provider to create the actual TinkerbellMachine resources that represent the physical servers in your cluster.

3. Define the TinkerbellMachine Resource

The TinkerbellMachine resource represents a single machine in your Kubernetes cluster. Here’s how you might define one:

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
kind: TinkerbellMachine
metadata:
  name: my-cluster-control-plane-1
  namespace: default
spec:
  providerID: "tinkerbell://my-cluster-control-plane-1"
  hardwareSelector:
    manufacturer: "Intel"
    plan: "NUC"
  osImage: "ubuntu-20.04"
  userDataSecret:
    name: my-cluster-userdata

Apply this machine resource:

kubectl apply -f tinkerbell-machine.yaml

The Infrastructure Provider will translate this into a Tinkerbell workflow that provisions an Intel NUC with the specified OS image and configuration.

4. Create a MachineDeployment for Scaling

If you want to create multiple machines of the same type (e.g., worker nodes), you can use a MachineDeployment resource that references the TinkerbellMachineTemplate:

apiVersion: cluster.x-k8s.io/v1alpha4
kind: MachineDeployment
metadata:
  name: my-cluster-worker-md
  namespace: default
spec:
  replicas: 3
  clusterName: my-cluster
  selector:
    matchLabels:
      cluster.x-k8s.io/cluster-name: my-cluster
  template:
    metadata:
      labels:
        cluster.x-k8s.io/cluster-name: my-cluster
    spec:
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1alpha4
          kind: KubeadmConfigTemplate
          name: my-cluster-bootstrap-template
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
        kind: TinkerbellMachineTemplate
        name: my-cluster-worker-template
      version: v1.21.1

Apply this machine deployment:

kubectl apply -f machine-deployment.yaml

The Infrastructure Provider will ensure that Tinkerbell provisions three Intel NUCs as worker nodes, configured according to the specified template.

5. Monitoring and Managing the Cluster

As the Infrastructure Provider works in the background, you can monitor the progress and state of your machines:

kubectl get machines -A
kubectl get clusters -A
kubectl get machinedeployments -A

These commands show you the status of the cluster and the individual machines, including whether they’ve been successfully provisioned and are ready to join the Kubernetes cluster.

Conclusion

The Infrastructure Provider in CAPT is a crucial component that bridges the gap between Kubernetes cluster management and Tinkerbell’s bare-metal provisioning capabilities. By abstracting the underlying hardware and handling the integration with Tinkerbell, the Infrastructure Provider allows you to manage bare-metal Kubernetes clusters with the same ease and flexibility as cloud-based clusters. Whether you’re deploying Kubernetes on Intel NUCs or any other bare-metal hardware, the Infrastructure Provider ensures that your infrastructure is provisioned, scaled, and managed according to the desired state defined in your Kubernetes resources.