Out-of-Bounds Read in NVIDIA Triton Inference Server TensorRT Backend Leading to Denial of Service
Overview
A high-severity vulnerability was discovered in the NVIDIA Triton Inference Server, specifically within its TensorRT backend. The vulnerability, identified as an out-of-bounds read, can be triggered by a remote attacker sending a specially crafted inference request to a running model. The root cause lies in improper validation of tensor metadata within the request payload. An attacker can construct an input tensor with mismatched dimension and size properties. When the Triton server's TensorRT backend processes this malformed request, it fails to perform adequate boundary checks before attempting to read from the memory buffer associated with the tensor. This results in an attempt to read memory outside the allocated buffer, which causes the Triton server process to crash immediately. This crash leads to a denial of service (DoS) for all models hosted on that server instance, disrupting any AI applications that rely on it. Successful exploitation requires network access to the server's inference port (HTTP/S or gRPC). While the vulnerability does not lead to code execution, the ability for a low-complexity, unauthenticated attack to cause service disruption makes it a significant threat to production MLOps environments that depend on Triton for high-availability model serving.
Affected Systems
Testing Guide
1. Check the running version of your Triton Inference Server. This can often be found in startup logs or by querying the server's metadata endpoint. 2. Construct a proof-of-concept client script that sends an inference request with a tensor whose metadata (e.g., shape, datatype, byte_size) is intentionally inconsistent. 3. Send the request to a model running with the TensorRT backend on a non-production server instance. 4. Observe if the Triton server process terminates unexpectedly. If it does, the instance is vulnerable.
Mitigation Steps
1. Upgrade the NVIDIA Triton Inference Server to version 2.65.0 or later, which contains the patched TensorRT backend. 2. If an immediate upgrade is not possible, place a reverse proxy or API gateway in front of the Triton server to perform stricter input validation on inference requests before they reach the server. 3. Restrict network access to Triton inference endpoints to only trusted clients and internal networks. 4. Implement health checks and automated restart policies for the Triton server container or process to minimize downtime in case of a crash.
Patch Details
Upgrade to NVIDIA Triton Inference Server version 2.65.0. The patch includes enhanced validation logic in the TensorRT backend to correctly verify tensor metadata and reject malformed requests.