NVIDIA Triton Inference Server Path Traversal via Malicious Model Repository
Overview
A high-severity path traversal vulnerability was identified in the NVIDIA Triton Inference Server. The vulnerability, designated CVE-2024-0081, allows an attacker with permissions to manage models in a model repository to read or write arbitrary files on the server's filesystem. The flaw exists in how the server handles model loading from local storage paths. An attacker can craft a model configuration file (`config.pbtxt`) or use a specially named model file that includes '..' path traversal sequences. When the Triton server attempts to load this malicious model, it resolves the path incorrectly, allowing access to files outside the intended model repository directory. A successful exploit could lead to the exfiltration of sensitive data, such as other ML models, configuration files, or system credentials. Furthermore, an attacker could overwrite critical system files or other models, leading to a denial of service or the execution of a poisoned model. This vulnerability affects multi-tenant environments where different users or teams manage their own models, as a malicious user in one tenant could potentially access or corrupt the data of another.
Affected Systems
Testing Guide
1. **Setup Test Environment:** Deploy a vulnerable version of Triton Inference Server (e.g., 23.12) and configure a model repository. 2. **Create Malicious Model:** Create a new model directory (e.g., `malicious_model`) within the repository. Inside this directory, create a symbolic link or a configuration file that uses a path traversal sequence to point to a known sensitive file outside the repository, such as `/etc/passwd`. - Example: `ln -s ../../../../../../../etc/passwd 1/model.pt` 3. **Load the Model:** Use the Triton client API to issue a model load request for `malicious_model`. 4. **Verify Access:** Attempt to access the linked file through the server or check if the server logs show an error or unauthorized access attempt. A successful exploit would allow reading the contents of `/etc/passwd`.
Mitigation Steps
1. **Update Triton Server:** Upgrade to the patched version of Triton Inference Server (24.01 or later) as specified in the NVIDIA security bulletin. 2. **Restrict Repository Access:** Limit write permissions to the model repository to only trusted administrators. Implement strict access control lists (ACLs) on the repository directory. 3. **Use Secure Mounts:** When running Triton in a container, mount the model repository using read-only flags if models are not intended to be changed dynamically at runtime. 4. **Audit Model Configurations:** Regularly scan model repository configurations for suspicious path traversal sequences (`../`, `..\`) or absolute paths.
Patch Details
The vulnerability is addressed in NVIDIA Triton Inference Server version 24.01 and later.