docs: add AMD GPU Flex Node setup guidance#230
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an AMD GPU (ROCm) Flex Node lab guide for AMD Instinct hosts, and updates the docs index to distinguish NVIDIA vs AMD GPU setup guidance. This aligns the documentation with the “prepared-host” contract where GPU driver installation and the cluster GPU stack are explicitly outside aks-flex-node start.
Changes:
- Renames the existing GPU lab to NVIDIA GPU Flex Node setup and updates index entries accordingly.
- Adds a new AMD GPU Flex Node setup lab covering validated Ubuntu 24.04 + MI300X host preparation and post-join cluster GPU stack options.
- Updates docs landing pages to link to both NVIDIA and AMD GPU lab guides.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| docs/usage.md | Splits the prior generic GPU lab link into explicit NVIDIA and AMD GPU setup entries. |
| docs/labs/README.md | Updates the labs index and topic matrix to include separate NVIDIA/AMD GPU lab entries. |
| docs/labs/gpu-node-setup.md | Retitles the existing GPU lab to NVIDIA-specific wording. |
| docs/labs/amd-gpu-node-setup.md | New AMD Instinct/ROCm prepared-host and cluster GPU stack runbook (validated path documented). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
bcho
left a comment
bcho
left a comment
There was a problem hiding this comment.
thanks for adding the new guide! A few comments
|
Addressed the AMD GPU setup review feedback in 2397daf and resolved all five threads. The guide now pins GPU Operator v1.5.0, cert-manager v1.15.1, device plugin v1.31.0.10, ROCm workload image 7.2.4, and AKSFlexNode v0.14; disables operator/KMM host-driver ownership; uses an explicit MI300X VF DeviceConfig; and adds an amd.com/gpu Pod diagnostic. Local validation: pinned URLs returned HTTP 200, the Helm chart rendered with the default DeviceConfig disabled, git diff --check passed, and make check passed. No authorized/reachable MI300X environment is currently available (no MI300X VM in the active subscription and saved cluster endpoints no longer resolve), so I did not claim end-to-end Flex Node/operator/8-GPU/Pod results or provision cost-bearing resources. Ready for re-review. |
Summary
Adds AMD GPU Flex Node setup guidance for AMD Instinct hosts, with the current validated path focused on Ubuntu 24.04 + Standard_ND96isr_MI300X_v5 + ROCm 7.2.4.
The doc mirrors the existing NVIDIA GPU Flex Node contract: Flex Node joins a prepared host, while GPU driver preparation and the cluster GPU stack remain outside
aks-flex-node start.Details
repo.radeon.comat node boot.amd.com/gpuusage.Validation
Validated on a clean Azure VM:
Standard_ND96isr_MI300X_v5francecentral6.17.0-1018-azure7.2.430.30.4Validation performed:
/opt/rocmoramdgpu-dkmsinstallation.amdgpu/6.16.13-2341068.24.04for6.17.0-1018-azure./dev/kfdand render nodes were present.rocm-smidetected 8AMD Instinct MI300X VFdevices.rocminfodetected 8AMD Instinct MI300X VFentries and 8gfx942entries.blacklist amdgpuissue; after removing that blacklist entry, PCI device discovery loadsamdgpuafter reboot without forcing it through/etc/modules-load.d.repo.radeon.comapt sources were removed after installation.git diff --check.Notes
Other AMD GPU families, OS versions, kernels, and package streams should be treated as separate validation targets before using this as a production runbook.