Path Traversal in NVIDIA Triton Inference Server Allows Arbitrary File Read
Overview
A high-severity path traversal vulnerability was discovered in the NVIDIA Triton Inference Server. The vulnerability, identified as CVE-2024-0081, allows an attacker with privileges to manage the model repository to read arbitrary files from the server's filesystem. The flaw exists in the model loading mechanism, which fails to properly sanitize the file paths specified within a model's `config.pbtxt` file. An attacker can craft a model configuration that uses directory traversal sequences (`../`) in the path to a model's data files. When Triton is instructed to load this malicious model via its API, it resolves the path and reads a file from outside the intended model repository directory. This can be exploited to exfiltrate sensitive information from the host system, such as `/etc/passwd`, SSH private keys, cloud credentials, or proprietary application code. The exploit requires that the Triton server is configured to allow dynamic model loading and that the attacker has network access to the Triton management port. The vulnerability underscores the importance of securing the MLOps infrastructure pipeline, as a compromise in the model management layer can lead to a full breach of the host system.
Affected Systems
Testing Guide
To test if your Triton instance is vulnerable: 1. Create a directory for a fake model, e.g., `malicious_model/1/`. 2. Inside `malicious_model`, create a `config.pbtxt` file with the following content, replacing `../../../../etc/passwd` with a path to a known file: ``` name: "malicious_model" platform: "onnxruntime_onnx" default_model_filename: "../../../../../../etc/passwd" input [{name: "INPUT0", data_type: TYPE_FP32, dims: [1]}] output [{name: "OUTPUT0", data_type: TYPE_FP32, dims: [1]}] ``` 3. Place this directory in your model repository. 4. Send a request to load the model: `curl -X POST localhost:8001/v2/repository/models/malicious_model/load`. 5. If the server returns a 200 OK status, the model loaded the target file, and the server is vulnerable. Check server logs for errors related to parsing the file as a model.
Mitigation Steps
1. **Upgrade Triton**: Immediately upgrade the NVIDIA Triton Inference Server to version 24.01 or later. 2. **Restrict Access**: Limit network access to the Triton Inference Server's management ports (e.g., 8000, 8001, 8002) to trusted administrative hosts only. 3. **Disable Dynamic Loading**: If not required, disable the dynamic model loading feature by running Triton with the `--model-control-mode=NONE` flag. 4. **Run as Non-Root**: Operate the Triton server using a dedicated, low-privilege user account to limit the scope of files that can be accessed in the event of a compromise.
Patch Details
This vulnerability is addressed in NVIDIA Triton Inference Server version 24.01. The fix involves implementing robust path validation and sanitization logic to prevent directory traversal.