Denial of Service in NVIDIA CUDA Driver via Malformed Compute Kernels
Overview
A medium-severity denial-of-service (DoS) vulnerability was discovered in the NVIDIA GPU driver for both Linux and Windows. The vulnerability allows a low-privileged user with access to the CUDA API to trigger a GPU hardware fault, resulting in a complete freeze of the affected GPU. This condition, often referred to as a 'GPU hang,' renders the device unresponsive to any further commands and can only be resolved by a hard reboot of the host machine. The issue was traced to improper handling of certain complex memory access patterns within a custom-written compute kernel. An attacker could craft a malicious CUDA application that, when executed, would trigger this fault condition reliably. In a multi-tenant environment, such as a shared high-performance computing (HPC) cluster or a cloud-based ML training platform (e.g., AWS SageMaker, GCP Vertex AI with GPU instances), a single malicious tenant could exploit this vulnerability to disrupt the workloads of all other users sharing the same physical GPU or server. While the vulnerability does not lead to code execution or data leakage, it poses a significant availability risk for mission-critical AI training and inference workloads.
Affected Systems
Testing Guide
1. Verify the installed NVIDIA driver version using the `nvidia-smi` command. 2. If the version is within the affected range, attempt to obtain the proof-of-concept (PoC) code from the vulnerability disclosure source. 3. Compile and run the PoC in a non-production environment. A vulnerable system will experience a GPU hang, and the `nvidia-smi` command will become unresponsive or report an error.
Mitigation Steps
1. Update NVIDIA drivers to the patched versions (550.78 for Linux, 552.61 for Windows) or newer. 2. In multi-tenant environments, consider using GPU virtualization technologies that provide better isolation between tenants. 3. Implement monitoring and alerting for GPU health metrics (e.g., via `nvidia-smi`) to quickly detect unresponsive GPUs. 4. Apply resource quotas and execution time limits to user-submitted jobs to limit the potential impact of a malicious or faulty kernel.
Patch Details
NVIDIA released patched drivers in their September 2025 security bulletin.