Path Traversal in NVIDIA Triton Inference Server Model Repository Allows Arbitrary File Read
Overview
A high-severity path traversal vulnerability was found in the NVIDIA Triton Inference Server. The vulnerability exists in the server's handling of model loading from a model repository. An authenticated attacker with permissions to add or modify models in the repository could create a specially crafted model configuration file (`config.pbtxt`). By using '..' sequences in file path parameters within this configuration, the attacker could trick the server into loading files from outside the intended model directory. This allows the attacker to read arbitrary files on the server's filesystem that are accessible to the user running the Triton server process. The impact is significant, as it could lead to the exfiltration of sensitive data, including other models' weights, configuration files, environment variables, or system credentials like SSH keys. The vulnerability was reported to NVIDIA by researchers from the NCC Group and was patched in a subsequent security update. This issue underscores the importance of input sanitization even in server components that are typically accessed by trusted users, as privilege escalation or insider threats are serious risks.
Affected Systems
Testing Guide
1. Set up a Triton server using a vulnerable version (e.g., NGC container 24.03 or older). 2. Create a model directory in the model repository. 3. Inside the model directory, create a `config.pbtxt` file with a parameter that points to a sensitive file on the server, for example: `default_model_filename: "../../../../../../etc/passwd"`. 4. Attempt to load the model via the Triton API. If the server throws an error message containing the contents of `/etc/passwd` or successfully loads it, the server is vulnerable. The patched version should return an error indicating an invalid or inaccessible path.
Mitigation Steps
1. Upgrade NVIDIA Triton Inference Server to version 2.45.0 (included in the NGC 24.04 container) or later. 2. Restrict access to the model repository. Only highly trusted users and automated CI/CD pipelines should have write permissions. 3. Run the Triton server as a non-root user with minimal privileges to limit the scope of what files an attacker can access if the vulnerability is exploited. 4. Use a filesystem monitoring tool to detect suspicious file access patterns originating from the Triton server process.
Patch Details
Patched in Triton Inference Server version 2.45.0, which is part of the monthly NVIDIA NGC container releases (version 24.04 and later).