NVIDIA Triton Inference Server TensorRT-LLM Backend Buffer Overflow
Overview
NVIDIA released a security bulletin for a high-severity heap-based buffer overflow vulnerability in the TensorRT-LLM backend for the Triton Inference Server. An unauthenticated remote attacker could trigger this vulnerability by sending a specially crafted inference request with malformed metadata for an input tensor. By manipulating properties such as the tensor's shape or data type, an attacker could cause the backend to miscalculate the required memory allocation size during request processing. This leads to a buffer overflow when the tensor data is copied. Successful exploitation can reliably cause a denial of service (DoS) by crashing the Triton server process. Furthermore, NVIDIA's advisory stated that this vulnerability could potentially be leveraged for arbitrary code execution on the host machine with the permissions of the Triton server process. Given that Triton servers are often deployed in production environments with direct access to powerful GPUs and sensitive data, a successful RCE attack would be devastating. This flaw underscores the critical need for robust input validation even at the infrastructure level of the MLOps stack. The vulnerability affects deployments that utilize the TensorRT-LLM backend for high-performance LLM inference.
Affected Systems
Testing Guide
1. Check the version of your running Triton Inference Server. You can often find this in container logs or by querying the server's metadata endpoint (`/v2/servermetadata`). 2. If the `version` field shows a number lower than `2.45.0`, you are likely affected. 3. Use a security scanner with a plugin for this CVE to probe the endpoint. Note: A public proof-of-concept exploit might not be available or safe to run. 4. The safest test is to verify the installed software versions against the patched versions listed in the NVIDIA security bulletin.
Mitigation Steps
1. Upgrade the NVIDIA Triton Inference Server to version `2.45.0` or later by using the latest NGC container (e.g., `nvcr.io/nvidia/tritonserver:24.05-py3`). 2. Ensure the associated TensorRT-LLM backend is also updated to version `0.9.0` or later. 3. Restrict network access to the Triton inference endpoints (e.g., ports 8000, 8001, 8002) to only trusted clients using firewall rules or security groups. 4. Run Triton Inference Server with the least privilege possible and within a containerized, isolated environment.
Patch Details
Patched in Triton Inference Server container versions 24.05 and later, which includes the updated TensorRT-LLM backend.