A new version of ROCm has been released that can better utilize AMD AI Max 395
The existing version was mainly ROCm for graphics cards
This time it seems to be preparing for 495
Since an official version supporting ROCm hasn't been released in vllm or llama yet
I asked my agent to build one for llama
Since vllm would take several hours, I chose llama, and seeing it finish earlier than expected, vllm might not take that long either, but for now I've decided to try the official version when it comes out
Previously, Vulkan had better performance than ROCm
But this time I tested it with ROCm version 10
What was previously lagging behind Vulkan now seems to be similar or slightly improved
Since this was measured on Linux, Windows results may differ
https://rocm.docs.amd.com/en/latest/about/release-notes.html
I gave my agent this URL and cast the spell of having them build it with llama and handle it themselves
