Linux Kernel Upgrade Causing NVIDIA Driver Failure

Last Updated: Nov 5, 2025

Overview

Some applications require a specific Linux kernel version to function correctly.
When upgrading the Linux kernel on a VM running NVIDIA drivers, driver failures may occur if the new kernel is not compatible with the installed NVIDIA or CUDA version.

Prerequisites

• SSH access to the VM
• Root privilege

Step-by-Step Instructions

1. Identify the Installed CUDA Version

   • For Traditional VMs
     Run the following command on your VM:

     $ nvidia-smi 

   • For Kubernetes Nodes
     Run the following command to query the CUDA version:

      $ kubectl -n gpu-operator exec ds/nvidia-driver-daemonset -- nvidia-smi

2. Identify the Supported Kernel Version

   • Visit the official NVIDIA CUDA Toolkit Archive
   • Locate your CUDA version from the list.
   • Click the “Versioned Online Documentation” link next to it.
   • Under Installation Guides, open the guide for Linux.
   • In the System Requirements section, find the supported kernel version for your Linux distribution.
      

3. Align Kernel Version with CUDA Requirements

   To upgrade or downgrade your Linux kernel to the recommended kernel version, run:

   $ sudo apt-get install linux-image-x.x.x-x-generic linux-headers-x.x.x-x-generic

   📝 Replace x.x.x-x with the kernel version identified in Step 2.

   Additional Resources

   • NVIDIA CUDA Toolkit Archive