How-to Configure maxPods in CMK Worker Nodes

Last Updated: Dec 26, 2025

This article describes the procedure to increase the maximum number of pods (maxPods) that can run on a Crusoe Managed Kubernetes (CMK) worker node. This can resolve issues where workloads fail to schedule due to reaching the pod limit, even when other resources like CPU, memory, and GPUs are available.

Pre-Requirements

• Access to a Kubernetes cluster created through Crusoe Managed Kubernetes (CMK).
• kubectl access to the CMK cluster.
• SSH access to the individual worker nodes within the cluster.

Important Considerations

Currently, CMK nodepools do not support startup scripts or custom node configurations upon creation. The following procedure is a manual workaround that must be applied to each worker node individually.

Note: If a worker node is replaced for any reason (e.g., through an autoscaling event or manual replacement), these manual changes will be lost. The procedure must be performed again on the new node.

Steps to Increase maxPods

1. Identify the Pod Limit

   First, verify if your node has reached its maximum pod capacity. Use kubectl to describe the node and check the pods value under Capacity and Allocatable, as well as the list of Non-terminated Pods.

   $ kubectl describe node <node_name>

   Example:

   In the output below, the node has a capacity of 110 pods and is currently running 110 pods, indicating it cannot schedule any new ones.

    

   $ kubectl describe node np-0344dfdb-3.us-southcentral1-a.compute.internal
   Name:               np-0344dfdb-3.us-southcentral1-a.compute.internal
   ...
   Capacity:
     ...
     pods:             110
   Allocatable:
     ...
     pods:             110
   ...
   Non-terminated Pods: (110 in total)
   ...
   

2. Connect to the Worker Node

   Use SSH to log into the worker node where you need to increase the pod limit.

3. Modify the Kubelet Configuration File

   Open the kubelet configuration file for editing.

   $ sudo vi /etc/kubernetes/kubelet-config.yaml

   Add the maxPods parameter to the file with your desired value. For example, to increase the limit to 200:

    

   apiVersion: kubelet.config.k8s.io/v1beta1
   kind: KubeletConfiguration
   ...
   maxPods: 200
   ...
   

4. Restart the Kubelet Service

   After saving the configuration file, you must reload the systemd manager configuration and restart the kubelet service for the changes to take effect.

    

   $ sudo systemctl daemon-reload
   $ sudo systemctl restart kubelet
   

5. Verify the New Limit 

   From your local machine, describe the node again to confirm that the pod capacity has been updated.

   $ kubectl describe node <node_name>

   Example:

   The output should now reflect the new maxPods value.

    

   $ kubectl describe node np-0344dfdb-3.us-southcentral1-a.compute.internal
   Name:               np-0344dfdb-3.us-southcentral1-a.compute.internal
   ...
   Capacity:
     ...
     pods:             200
   Allocatable:
     ...
     pods:             200
   ...
   

   Repeat these steps for any other worker nodes that require a higher pod limit.

Additional Resources

• Kubernetes - Kubelet configuration