Heap-based Buffer Overflow in NVIDIA Triton Inference Server's TensorRT-LLM Backend
Overview
A high-severity heap-based buffer overflow vulnerability was found in the NVIDIA Triton Inference Server when using the TensorRT-LLM backend. The vulnerability can be triggered by sending a specially crafted inference request containing a malformed tensor shape definition. An attacker can specify tensor dimensions that lead to an integer overflow during memory allocation calculation within the backend's request-handling code. This results in a smaller-than-expected buffer being allocated on the heap. When the server subsequently tries to copy the request's tensor data into this undersized buffer, a heap overflow occurs. A remote, unauthenticated attacker can exploit this vulnerability by sending a malicious request to an exposed Triton Inference Server endpoint. Successful exploitation could lead to a denial of service (DoS) by crashing the server process. More sophisticated exploits could potentially achieve remote code execution (RCE) under the security context of the Triton server, allowing an attacker to compromise the inference infrastructure, access or poison models, and potentially move laterally within the MLOps network. The issue affects cloud and on-premise deployments that expose the Triton server's gRPC or HTTP endpoints to untrusted clients.
Affected Systems
Testing Guide
1. Check the version of your running Triton Inference Server. If it is below 2.47.0 and uses the TensorRT-LLM backend, it is potentially vulnerable. 2. Using a network security scanner, probe the Triton server with a proof-of-concept request containing oversized tensor dimension values. 3. Monitor the Triton server logs and process status. If the server process crashes or becomes unresponsive after receiving the test request, it is confirmed to be vulnerable.
Mitigation Steps
1. Upgrade the NVIDIA Triton Inference Server to version `2.47.0` or later. 2. If an immediate upgrade is not possible, restrict network access to the Triton Inference Server endpoints (default ports 8000, 8001, 8002) to only trusted clients using firewall rules or a reverse proxy with IP allowlisting. 3. Implement robust input validation on a front-end API gateway to check for reasonable tensor dimensions and request sizes before forwarding requests to the Triton server.
Patch Details
NVIDIA released Triton Inference Server version 2.47.0, which contains a patch that adds proper integer overflow checks and strengthens input validation in the TensorRT-LLM backend.