HIP/ROCm support for AMD GPUs — working port on RX 9070 (gfx1201)

[spheenik]

Hi there! 👋

I stumbled across the Dynamic VRAM blog post today and got really excited — this is exactly what I've been needing for my AMD setup. The post mentions AMD support is planned for the future, but I couldn't wait, so I fired up my slop generator (Claude Code) and about an hour later it came back with a working HIP port.

What I did

The entire port lives in a single file change to src/plat.h — an #ifdef __HIP_PLATFORM_AMD__ block at the top that maps all CUDA Driver API types and functions to their HIP equivalents. All .c files remain completely unmodified.

Key points:

• CUdeviceptr kept as unsigned long long (matching CUDA) with cast macros for HIP APIs that expect void* — preserves all upstream pointer arithmetic
• cuMemAllocHost wrapped to supply HIP's extra flags parameter
• cuGetErrorString wrapped to adapt HIP's direct-return API style
• pyt-cu-plug-alloc-async.c excluded from the build (the cudaMalloc/cudaFree symbol interposition is not portable to HIP, but the VBAR path that ComfyUI uses doesn't depend on it)

Test results (RX 9070, gfx1201, ROCm 7.2)

All HIP VMM APIs work correctly:

• hipMemAddressReserve — zero VRAM cost, as expected
• hipMemCreate / hipMemMap / hipMemSetAccess — physical VRAM allocated and accessible
• hipMemUnmap / hipMemRelease — clean deallocation
• Full VBAR allocate → fault → unpin lifecycle works

ComfyUI startup log:

comfy-aimdo inited for GPU: AMD Radeon RX 9070 (VRAM: 16304 MB)
DynamicVRAM support detected and enabled

Workflows that previously crashed with OOM (multi-model, full 1 megapixel resolution) now complete successfully with dynamic weight eviction working as intended.

The fork

I've pushed the change to a branch on my fork: spheenik/comfy-aimdo@hip-rocm-support

Happy to open a PR if you're interested. I understand you might want to approach this differently for an official implementation (CI, wheel builds, etc.) — just wanted to share that the HIP VMM APIs are there and working on RDNA 4, in case it's useful for your roadmap.

Build instructions (for anyone who wants to try)

gcc -shared -fPIC -O2 -D__HIP_PLATFORM_AMD__ -Isrc \
    -I/opt/rocm/include -L/opt/rocm/lib \
    src/vrambuf.c src/model-vbar.c src/control.c src/hostbuf.c \
    src/debug.c src/pyt-cu-plug-alloc.c src-posix/model-mmap.c \
    -lamdhip64 -o aimdo.so

Then replace the CUDA aimdo.so inside the comfy_aimdo pip package directory and start ComfyUI with --enable-dynamic-vram.

Thanks for the great work on AIMDO — it's a game changer!

[spheenik]

A few clarifications to keep expectations honest:

What was actually tested:

• All HIP VMM APIs individually via a C smoke test (hipMemAddressReserve, hipMemCreate, hipMemMap, hipMemSetAccess, hipMemUnmap, hipMemRelease + readback verification)
• VBAR allocate → fault → unpin lifecycle through the Python ctypes bindings
• One real ComfyUI workflow that previously OOM'd — it completed at full resolution, which implies eviction is working (the model weights didn't fit in VRAM all at once)

What was NOT tested:

• Any GPU other than RX 9070 (gfx1201)
• Any ROCm version other than 7.2
• Long-running stability / memory leaks over many workflows
• The pyt-cu-plug-alloc-async.c path (excluded from build — the cudaMalloc symbol interposition doesn't have a straightforward HIP equivalent)

Build environment:

• Arch Linux host, gcc 15.1, ROCm 7.2 (hip-runtime-amd package)
• Built natively on the machine with the GPU — not tested in any CI or cross-compile setup

So this is very much a "works on my machine" proof of concept, but at least it shows the HIP VMM APIs are functional on RDNA 4 consumer hardware and the AIMDO architecture ports cleanly.

[woct0rdho]

Have you checked #2 ?

[spheenik]

I found it only afterwards. Sorry. But I did not have the problems mentioned in #2. And I found this approach to be less invasive, and wanted to share it anyway. FWIW.

I would be happy to improve this into something that also works on Windows, and does not monkey patch CUDA out.

[rattus128]

Yeah this is well discussed and underway in #2. Ill catch up soon!

[jnolck]

I don't think it's working as intended. If I enable it, it makes my generation times a bit longer on workflows that don't spill over. Maybe I'm doing something wrong. It compiled just fine and I manually moved the *.so over to the installation folder. '''Total VRAM 20464 MB, total RAM 63561 MB
pytorch version: 2.10.0+rocm7.13.0a20260330
Set: torch.backends.cudnn.enabled = False for better AMD performance.
AMD arch: gfx1100
ROCm version: (7, 13)
Set vram state to: NORMAL_VRAM
Disabling smart memory management
Device: cuda:0 AMD Radeon RX 7900 XT : native
Using async weight offloading with 2 streams
Enabled pinned memory 60382.0
DEBUG: C++ CK Backend successfully loaded and forced via env var.
Using CK Flash Attention
Python version: 3.12.13 (main, Mar 26 2026, 00:00:00) [GCC 16.0.1 20260321 (Red Hat 16.0.1-0)]
ComfyUI version: 0.18.1
comfy-aimdo version: 0.2.12
comfy-kitchen version: 0.2.8
ComfyUI frontend version: 1.42.8'''

[spheenik]

Thanks for testing this! Looking at your log output though, I don't think the HIP VMM backend is actually active in your setup.

When AIMDO's memory manager initializes successfully, it prints a line like this:

aimdo: control.c:68:INFO:comfy-aimdo inited for GPU: AMD Radeon RX 7900 XT (VRAM: 20464 MB)

On our end (RX 9070, ROCm 7.2), we see this message followed by DynamicVRAM support detected and enabled:

aimdo: control.c:68:INFO:comfy-aimdo inited for GPU: AMD Radeon RX 9070 (VRAM: 16304 MB)
DynamicVRAM support detected and enabled

Both lines are missing from your log, which means init() was never called — so the HIP VMM interposition isn't running. The slowdown you're seeing is likely coming from something else.

Could you double-check:

• Where exactly you placed the .so file?
• Whether you set any environment variables to enable the HIP backend?

Also worth noting: this was only tested on gfx1201 (RX 9070) with ROCm 7.2, so there may be additional steps needed on gfx1100. Happy to help debug further!

[jnolck]

I didn't notice that! I manually put mv aimdo.so ../../.venv/lib/python3.12/site-packages/comfy_aimdo/
aimdo.so host_buffer.py model_vbar.py torch.py
control.py model_mmap.py pycache/ _version.py . I copy pasted that after that crash, but it was enabled during it. I just tried to run a workflow at 1280x720, something I normally wouldn't try since I know it'll OOM.

'''Total VRAM 20464 MB, total RAM 63561 MB
pytorch version: 2.10.0+rocm7.13.0a20260330
Set: torch.backends.cudnn.enabled = False for better AMD performance.
AMD arch: gfx1100
ROCm version: (7, 13)
Set vram state to: NORMAL_VRAM
Device: cuda:0 AMD Radeon RX 7900 XT : native
Using async weight offloading with 2 streams
Enabled pinned memory 60382.0
DEBUG: C++ CK Backend successfully loaded and forced via env var.
Using CK Flash Attention (Forced)
aimdo: src/control.c:68:INFO:comfy-aimdo inited for GPU: AMD Radeon RX 7900 XT (VRAM: 20464 MB)
DynamicVRAM support detected and enabled
Python version: 3.12.13 (main, Mar 26 2026, 00:00:00) [GCC 16.0.1 20260321 (Red Hat 16.0.1-0)]
ComfyUI version: 0.18.1
comfy-aimdo version: 0.2.12
comfy-kitchen version: 0.2.8
ComfyUI frontend version: 1.42.8
[Prompt Server] web root: /home/jnolck/src/ComfyUI/.venv/lib64/python3.12/site-packages/comfyui_frontend_package/static
Asset seeder disabled'''
I did not set any env variables. If there's something I'm supposed to set let me know. I have no idea how any of this works, so I don't even know what to look at or look for.

[spheenik]

That looks much better — AIMDO is active now!

On my machine, this is exactly what made the difference: workflows that previously OOM'd ran through successfully with DynamicVRAM enabled. No extra env variables needed.

In my case, it always crashed during VAE decode. Since the HIP patch, AIMDO evicts older allocations at that point and the workflow completes cleanly.

Did the 1280x720 workflow complete this time?

[jnolck]

I ended up porting it myself because I thought there was something something wrong with your port and in the process discovered that only the standard comfyui nodes work with aimdo (until those other nodes wrap things around ModelVBAR, like the comfy ones do). I was using a gguf loader so aimdo wouldn't get activated. I couldn't use fp8 or fp16 standard models I'd have to use gguf q6 or less to not crash amdgpu. That's what happened. Now I can even run ltx 2.3.

Use standard ComfyUI nodes if you want to try this.

[asagi4]

Are you running this on Windows only? If not, this'll have the same memory leak issue that I found in my port unless you're using a fixed version of hipruntime already (the bug is not in AIMDO, but in the HIP runtime), but I don't think the fix is in a release yet.

[spheenik]

Good catch — Linux only, ROCm 7.2.1 on Arch.

I just ran a reproducer based on your test from ROCm/ROCm#6021 and can confirm: hipMemUnmap + hipMemRelease without hipMemAddressFree does not free any VRAM on my system. The 1 GiB stays allocated even though all API calls return hipSuccess. So yes, the AIMDO eviction path is silently broken here.

My original testing only covered single-workflow runs where eviction happened once (VAE decode), so I wouldn't have noticed the leak across multiple cycles.

The fix in rocm-systems#4363 was merged April 1st — do you know if it's in TheRock nightlies yet? I'm on the Arch rocm-hip-sdk package (7.2.1) so I'll need to either rebuild libhsa-runtime64 from the patched source or wait for it to land downstream.

[asagi4]

@spheenik building the hip runtime by yourself is not too difficult, but as a workaround you can just add a manual mmap call: 9719699
That does the same thing as the proper fix.