Summary
This issue was surfaced through llama.cpp #24906. On a Linux desktop, a game or other Vulkan/DRM client could hold several GiB of VRAM while the ROCm backend still saw nearly all VRAM as free through hipMemGetInfo. llama.cpp uses that value to size automatic GPU offload, so the over-report can lead to a late allocation failure or OOM.
Kernel root cause
hipMemGetInfo reaches HSA_AMD_AGENT_INFO_MEMORY_AVAIL, then KFD AMDKFD_IOC_AVAILABLE_MEMORY and amdgpu_amdkfd_get_available_memory(). The KFD calculation accounts for KFD allocations and pinned VRAM but not an unpinned VRAM BO resident through another DRM client.
A two-direction component test also shows that the DRM heap already observes KFD allocations because KFD BOs receive TTM VRAM resources. The missing direction is KFD accounting for non-KFD DRM residency.
Minimal evidence
On an RX 7900 GRE / gfx1100 with ROCm 7.2.1, reproduced on Ubuntu kernels 6.14 and 7.0 using a 256 MiB allocation:
| Allocation |
HSA/KFD available drop |
DRM heap available drop |
| libdrm VRAM BO |
0 B |
256 MiB |
| HSA VRAM allocation |
256 MiB |
256 MiB |
A local kernel accounting prototype made the libdrm arm reduce both values by 256 MiB in 3/3 runs while preserving immediate KFD recovery after an HSA free. No PR is being opened yet; pinned/display, SVM migration, APU, and XCP/memory-partition coverage still need to be completed.
Previous tracking
Summary
This issue was surfaced through llama.cpp #24906. On a Linux desktop, a game or other Vulkan/DRM client could hold several GiB of VRAM while the ROCm backend still saw nearly all VRAM as free through
hipMemGetInfo. llama.cpp uses that value to size automatic GPU offload, so the over-report can lead to a late allocation failure or OOM.Kernel root cause
hipMemGetInforeachesHSA_AMD_AGENT_INFO_MEMORY_AVAIL, then KFDAMDKFD_IOC_AVAILABLE_MEMORYandamdgpu_amdkfd_get_available_memory(). The KFD calculation accounts for KFD allocations and pinned VRAM but not an unpinned VRAM BO resident through another DRM client.A two-direction component test also shows that the DRM heap already observes KFD allocations because KFD BOs receive TTM VRAM resources. The missing direction is KFD accounting for non-KFD DRM residency.
Minimal evidence
On an RX 7900 GRE / gfx1100 with ROCm 7.2.1, reproduced on Ubuntu kernels 6.14 and 7.0 using a 256 MiB allocation:
A local kernel accounting prototype made the libdrm arm reduce both values by 256 MiB in 3/3 runs while preserving immediate KFD recovery after an HSA free. No PR is being opened yet; pinned/display, SVM migration, APU, and XCP/memory-partition coverage still need to be completed.
Previous tracking