Dynamo Support Matrix — NVIDIA Dynamo Documentation
Dynamo Support Matrix #
This document provides the support matrix for Dynamo, including hardware, software and build instructions.
See also: Feature Compatibility Matrix for backend-specific feature support (vLLM, TensorRT-LLM, SGLang).
Hardware Compatibility #
| CPU Architecture | Status |
|---|---|
| x86_64 | Supported |
| ARM64 | Supported |
GPU Compatibility #
If you are using a GPU, the following GPU models and architectures are supported:
| GPU Architecture | Status |
|---|---|
| NVIDIA Blackwell Architecture | Supported |
| NVIDIA Hopper Architecture | Supported |
| NVIDIA Ada Lovelace Architecture | Supported |
| NVIDIA Ampere Architecture | Supported |
Platform Architecture Compatibility #
Dynamo is compatible with the following platforms:
| Operating System | Version | Architecture | Status |
|---|---|---|---|
| Ubuntu | 22.04 | x86_64 | Supported |
| Ubuntu | 24.04 | x86_64 | Supported |
| Ubuntu | 24.04 | ARM64 | Supported |
| CentOS Stream | 9 | x86_64 | Experimental |
Note
Wheels are built using a manylinux_2_28-compatible environment and they have been validated on CentOS 9 and Ubuntu (22.04, 24.04).
Compatibility with other Linux distributions is expected but has not been officially verified yet.
Caution
KV Block Manager is supported only with Python 3.12. Python 3.12 support is currently limited to Ubuntu 24.04.
Software Compatibility #
Runtime Dependency #
| Python Package | Version | glibc version | CUDA Version |
|---|---|---|---|
| ai-dynamo | 0.8.1 | >=2.28 | |
| ai-dynamo-runtime | 0.8.1 | >=2.28 (Python 3.12 has known issues) | |
| NIXL | 0.8.0 | >=2.27 | >=11.8 |
Build Dependency #
The following table shows the dependency versions included with each Dynamo release:
| Dependency | main (ToT) | v0.8.0 | v0.7.1 | v0.7.0.post1 | v0.7.0 |
|---|---|---|---|---|---|
| SGLang | 0.5.7 | 0.5.6.post2 | 0.5.3.post4 | 0.5.3.post4 | 0.5.3.post4 |
| TensorRT-LLM | 1.2.0rc6.post1 | 1.2.0rc6.post1 | 1.2.0rc3 | 1.2.0rc3 | 1.2.0rc2 |
| vLLM | 0.13.0 | 0.12.0 | 0.11.0 | 0.11.0 | 0.11.0 |
| NIXL | 0.8.0 | 0.8.0 | 0.8.0 | 0.8.0 | 0.8.0 |
Note
main (ToT) reflects the current development branch.
Important
Specific versions of TensorRT-LLM supported by Dynamo are subject to change. Currently TensorRT-LLM does not support Python 3.11 so installation of the ai-dynamo[trtllm] will fail.
CUDA Support by Framework #
| Dynamo Version | SGLang | TensorRT-LLM | vLLM |
|---|---|---|---|
| Dynamo 0.8.0 | CUDA 12.9, CUDA 13.0 (🧪) | CUDA 13.0 | CUDA 12.9, CUDA 13.0 (🧪) |
| Dynamo 0.7.1 | CUDA 12.8 | CUDA 13.0 | CUDA 12.9 |
🧪 = Experimental
Cloud Service Provider Compatibility #
AWS #
| Host Operating System | Version | Architecture | Status |
|---|---|---|---|
| Amazon Linux | 2023 | x86_64 | Supported¹ |
Caution
There is a known issue with the TensorRT-LLM framework when running the AL2023 container locally with docker run --network host ... due to a bug in mpi4py. To avoid this issue, replace the --network host flag with more precise networking configuration by mapping only the necessary ports (e.g., 4222 for nats, 2379/2380 for etcd, 8000 for frontend).
Build Support #
Dynamo currently provides build support in the following ways:
-
Wheels: We distribute Python wheels of Dynamo and KV Block Manager:
-
New as of Dynamo v0.7.0: kvbm as a standalone implementation.
-
Dynamo Runtime Images: We distribute multi-arch images (x86 & ARM64 compatible) of the Dynamo Runtime for each of the LLM inference frameworks on NGC:
-
Dynamo Kubernetes Operator Images: We distribute multi-arch images (x86 & ARM64 compatible) of the Dynamo Operator on NGC:
- kubernetes-operator to simplify deployments of Dynamo Graphs.
-
Helm Charts: NGC hosts the helm charts supporting Kubernetes deployments of Dynamo:
-
Rust Crates:
Once you’ve confirmed that your platform and architecture are compatible, you can install Dynamo by following the instructions in the Quick Start Guide.