Denial of Service in NVIDIA Triton Inference Server via Malformed Model Configuration
Overview
A high-severity denial-of-service (DoS) vulnerability was identified in the NVIDIA Triton Inference Server. The vulnerability existed in the server's model parsing logic, specifically when handling TensorRT (`.plan`) model files. An attacker with privileges to upload models to the server's model repository could provide a specially crafted or corrupted model configuration or plan file. When the Triton server attempted to load this malicious model, an out-of-bounds read would occur during the parsing process. This memory error would cause the main `tritonserver` process to crash immediately. Because a single Triton instance often serves multiple models concurrently for various applications, this crash would result in a complete denial of service for all inference requests being handled by that server. This could disrupt critical AI-powered services relying on the server for real-time inference. The vulnerability could be triggered remotely if the attacker has API access to manage the model repository. The issue highlights the importance of robust input validation, even for seemingly internal configuration files, in critical AI serving infrastructure.
Affected Systems
Testing Guide
1. Set up a test instance of a vulnerable version of NVIDIA Triton Inference Server. 2. Create a malformed or truncated TensorRT plan file or an invalid `config.pbtxt`. 3. Place the malformed model in the server's model repository and trigger a model load request via the API or by restarting the server. 4. Monitor the `tritonserver` process. If the process crashes or exits with a segmentation fault, the instance is vulnerable.
Mitigation Steps
1. **Upgrade Triton Server:** Update to NVIDIA Triton Inference Server version 23.04 or a later patched version. 2. **Restrict Model Repository Access:** Implement strict access control on the model repository. Only trusted, authenticated administrators should have write access. 3. **Validate Configurations:** Before deploying models to Triton, use validation scripts to check the integrity and correctness of the model configuration files (`config.pbtxt`) and associated artifacts. 4. **Implement High Availability:** Run multiple Triton instances in a load-balanced, high-availability configuration to minimize the impact of a single instance crashing.
Patch Details
The vulnerability is addressed in NVIDIA Triton Inference Server version 23.04 and all subsequent releases. The patch enhances the server's parsing logic with additional bounds checking and error handling for model artifacts.