#NVIDIA Triton Inference Server

También: triton inference server

Open-source server for deploying models from PyTorch, TensorRT, ONNX, and other backends. Covers dynamic batching, model ensembles, concurrent execution on GPUs, and metrics for production inference.