NVIDIA Brings TensorRT Multi-Device Inference to Dynamo-Triton for Multi-GPU Model Serving

It is a new capability that lets a single TensorRT network run distributed across multiple GPUs using NCCL collectives while retaining TensorRT inference optimizations. It is supported starting with TensorRT 11.0.

Previously teams had to split models manually or build custom orchestration layers. Now TensorRT and NCCL handle distribution inside the same inference path, reducing complexity while preserving optimized performance.

Yes, especially for organizations serving large generative models on on-prem or cloud GPU clusters, since it reduces deployment endpoints and simplifies horizontal scaling.

TensorRT 11.0 or later, a Dynamo-Triton environment, and a high-speed inter-GPU fabric such as NVLink or InfiniBand for best performance.