Heap Buffer Overflow in NVIDIA Triton Inference Server via Malicious TensorRT Model
Overview
Researchers from Google's Project Zero disclosed a high-severity heap buffer overflow vulnerability in NVIDIA's Triton Inference Server. The flaw exists in the server's model parsing and loading component, specifically when handling models optimized with TensorRT. An attacker can craft a malicious `.plan` model file containing a tensor definition with inconsistent metadata. When the Triton server attempts to load this model for inference, it incorrectly calculates the memory buffer size required for a specific layer's weights, leading to an out-of-bounds write on the heap. This memory corruption reliably crashes the inference server process, causing a Denial of Service (DoS) for all other models and users on that instance. While the primary impact is DoS, the researchers noted that with further effort, the heap overflow could potentially be exploited for remote code execution (RCE), which would allow an attacker to take full control of the inference server host. The vulnerability is particularly dangerous in multi-tenant environments or model hubs that allow users to upload their own models for serving, as it provides a direct path to compromising the underlying GPU infrastructure. NVIDIA was notified under a 90-day deadline and has released a patched version.
Affected Systems
Testing Guide
1. Obtain the proof-of-concept malicious `.plan` model file from the official vulnerability disclosure report. 2. Set up a vulnerable version of the NVIDIA Triton Inference Server in an isolated test environment. 3. Attempt to load the malicious model via the Triton API. 4. Observe if the `tritonserver` process crashes or becomes unresponsive. A crash indicates that your instance is vulnerable.
Mitigation Steps
1. **Upgrade Triton Server:** Immediately upgrade to NVIDIA Triton Inference Server version 2.45.0 or newer. 2. **Vet Models:** Do not load or serve models from untrusted or unverified sources. Implement a model scanning and validation pipeline before deploying models to production. 3. **Principle of Least Privilege:** Run the Triton server process as a non-root user with the minimum required permissions. 4. **Container Security:** Use up-to-date, minimal base container images for Triton and employ container scanning tools to detect known vulnerabilities in OS packages and libraries.
Patch Details
The vulnerability is addressed in NVIDIA Triton Inference Server container version 24.05 and later, corresponding to Triton version 2.45.0.