Heap Buffer Overflow in NVIDIA Triton Inference Server's TensorRT-LLM Backend
Overview
A critical heap buffer overflow vulnerability was discovered in the TensorRT-LLM backend of the NVIDIA Triton Inference Server. The flaw can be triggered by sending a specially crafted inference request containing a tensor with manipulated, invalid dimension metadata. When the Triton server attempts to process this malformed request in a batched operation, its memory allocation logic for the input tensor fails to perform adequate boundary checks. This leads to a write operation beyond the allocated buffer on the heap. Successful exploitation results in a denial of service (DoS) by crashing the inference server process, disrupting all ongoing and subsequent model inference tasks. Security researchers have indicated that under specific, complex memory layout conditions, it may be possible for an attacker to achieve arbitrary code execution with the permissions of the user running the Triton server. The vulnerability affects multi-tenant environments where untrusted users can submit inference requests, as well as public-facing AI services built on the Triton platform. NVIDIA has released a security bulletin and patched versions of the server software to address this issue.
Affected Systems
Testing Guide
1. **Check Version**: Verify the installed version of the Triton Inference Server. You can often find this in the startup logs or by querying the server's metadata endpoint. 2. **Craft Test Payload**: Using a client like `tritonclient`, construct an inference request for a deployed TensorRT-LLM model. Modify the request payload to specify an input tensor shape that is inconsistent with the tensor's actual data size. 3. **Send Request**: Send the malicious request to the server. 4. **Observe Behavior**: If the server process crashes or becomes unresponsive, it is likely vulnerable. Monitor server logs for memory-related errors or segmentation faults.
Mitigation Steps
1. **Upgrade Immediately**: Update the NVIDIA Triton Inference Server to version `2.45.0` or later and the TensorRT-LLM backend to `v0.10.0` or later. 2. **Input Validation**: If immediate patching is not possible, implement a validation layer in front of the Triton server to strictly check tensor shapes and metadata against the model's expected input schema. Reject any requests with mismatched dimensions. 3. **Isolate Workloads**: Run the Triton server in a container with minimal privileges and strict resource limits (e.g., memory, CPU) to reduce the impact of a potential crash or exploit. 4. **Network Segmentation**: Restrict network access to the Triton server's gRPC/HTTP ports, allowing connections only from trusted application frontends.
Patch Details
Patched in NVIDIA Triton Inference Server v2.45.0 and container image `nvcr.io/nvidia/tritonserver:26.06-py3`.