NVIDIA Triton Inference Server Remote Code Execution via Malicious Model Configuration
Overview
A critical vulnerability was discovered in the NVIDIA Triton Inference Server, a widely used solution for deploying and scaling AI models in production. The vulnerability, identified as CVE-2025-7359, allowed a remote, unauthenticated attacker to achieve arbitrary code execution on the server. The flaw resided in the server's handling of model configuration files (`config.pbtxt`) when loading a model from a repository. Triton allows models to specify custom backend shared libraries. An attacker with the ability to upload a model to a repository monitored by Triton (a common MLOps practice) could provide a malicious `config.pbtxt` file. This file could point to a specially crafted shared library file (a `.so` file), also uploaded by the attacker as part of the model's assets. When Triton loaded this model, it would load and execute the code within the malicious shared library with the permissions of the Triton server process. This could lead to a complete compromise of the inference server, data exfiltration of sensitive models and data, or lateral movement within the production Kubernetes cluster. The vulnerability affects all deployment modes, including local, containerized, and cloud-based setups.
Affected Systems
Testing Guide
1. Check the version of the Triton Inference Server you are running. If it is below 2.58.0, you are vulnerable. 2. In a non-production environment, create a model repository. 3. Create a simple C++ shared library (`.so`) that, for example, writes a file to `/tmp/pwned` upon loading. 4. Create a `config.pbtxt` file that specifies a custom backend pointing to your malicious shared library. 5. Upload the model (including the config and the `.so` file) to the repository and instruct Triton to load it. 6. Check if the file `/tmp/pwned` was created on the server. If so, the instance is vulnerable.
Mitigation Steps
1. Upgrade the NVIDIA Triton Inference Server to version 2.58.0 or later. 2. Strictly control access to model repositories. Only trusted, authenticated users should have write permissions. 3. Implement a model scanning and validation pipeline that checks `config.pbtxt` files for suspicious entries, such as custom backends, before they are loaded by Triton. 4. Run the Triton server with the least privilege possible, for example, as a non-root user and within a sandboxed environment like gVisor or Kata Containers. 5. Use network policies to restrict outbound traffic from the Triton server pods to prevent C2 communication.
Patch Details
Version 2.58.0 and later includes stricter validation of model configuration files and adds an option to disable custom backends entirely.