Introduction

By default, a Crusoe Managed Kubernetes (CMK) cluster is configured with a pod CIDR range of /16 and a node CIDR mask of /24. This setup allows for a total of 65,536 pod IP addresses. Since each node is allocated a /24 subnet (providing 256 IPs), the cluster can support a maximum of 256 nodes (65,536/256).

This article provides steps to increase maximum node count on existing CMK cluster to avoid the issue of insufficient IP address allocation in the pod network CIDR.

Prerequisites

Before starting, ensure you have the following:

• Access to a Crusoe Cloud project with appropriate permissions
• A CMK cluster with the latest image version (>= 1.30.8-cmk.26)
• Helm Client Installed
• Kubeconfig to access your CMK cluster

Step-by-Step Instructions

• Get the current Cilium configuration for the cluster.

$ helm --kubeconfig <kube-config-yaml>.yaml get values -n kube-system cilium > cilium-values.yaml

• Modify the contents of the file cilium-values.yaml to add an extra IPv4 Pod CIDR list.

MTU: 8900
bpf:
  masquerade: true
envoy:
  enabled: false
  prometheus:
    enabled: false
ipam:
  mode: cluster-pool
  operator:
    clusterPoolIPv4MaskSize: 24
    clusterPoolIPv4PodCIDRList:
    - 10.234.0.0/16
    - 10.235.0.0/16 //// NEW ADDition
ipv4:
  enabled: true
ipv4NativeRoutingCIDR: 10.234.0.0/16
ipv6:
  enabled: false
k8sServiceHost: <cluster-dns>
k8sServicePort: 443
kubeProxyReplacement: true
prometheus:
  enabled: true

Note: In above example, adding another pool of /16 PodCIDR List will ensure that the CMK cluster can now support upto 512 nodes in the cluster. Adjust this number as per requirement.

• Perform Helm upgrade to Cilium to apply new configuration.

$ helm --kubeconfig <kube-config-yaml>.yaml upgrade -n kube-system cilium/cilium --values ./cilium-values.yaml --version 1.16.1

• Restart Cilium Operator Deployment to ensure the CMK cluster picks up the new configuration.

$ kubectl rollout restart -n kube-system deploy/cilium-operator --kubeconfig <kube-config-yaml>.yaml

Additional Resources

• Kubernetes Documentation
• Cilium IPAM - Expanding Cluster Pool