Running Ollama on AMD Ryzen AI Max+ 395 with ROCm and Docker

2026-11-08来源 dev.toEnglish

Running large language models locally on an AMD integrated GPU takes more than installing Ollama. The operating system, ROCm runtime, container configuration, and GPU permissions all need to work together. A container can start successfully while inference still runs on the CPU—or fails when the model is loaded.

I set up Ollama on an EVO-X2 equipped with an AMD Ryzen AI Max+ 395 and Radeon 8060S, using Ubuntu, ROCm, Docker, and Open WebUI. This article documents that setup, how I checked GPU detection, and the compatibility problems I encountered later. It is based on my installation notes from late 2025, with later observations clearly distinguished from the initial procedure. It is not a new benchmark or a freshly validated installation on the latest software releases.

Hardware and software

My system has the following hardware:

Component	Configuration
System	EVO-X2
Processor	AMD Ryzen AI Max+ 395
Integrated GPU	AMD Radeon 8060S (RDNA 3.5, `gfx1151`)
System memory	128 GB unified system memory
Operating system	Ubuntu 24.04 series
Inference server	Ollama in Docker, using its ROCm image
Web interface	Open WebUI in a separate container

The original notes used floating container tags and did not record a complete, reproducible set of package versions. In subsequent work, I used the ROCm 7.1.1 installer and, in a March 2026 snapshot, confirmed Ollama 0.15.6. These observations do not establish the exact versions used at every stage of the original setup.

AMD's compatibility matrix now explicitly lists the Ryzen AI Max+ 395 and its gfx1151 GPU target. However, supported Ubuntu releases and kernel versions vary by ROCm release. Check the AMD ROCm compatibility matrix for the intended installation rather than copying an old kernel or driver version without checking it.

1. Prepare Docker

Docker Engine and the Docker Compose plugin are required. On a machine where Docker is already installed, verify both components:

docker --version docker compose version

If they are missing, follow the official Docker Engine installation guide for Ubuntu. Installing a Compose package alone does not necessarily install and configure Docker Engine.

To run Docker without sudo, I added my account to the docker group:

sudo usermod -aG docker "$USER"

Log out and sign in again before retrying Docker commands. Membership in the docker group grants root-level privileges; it should only be assigned to trusted users. See Docker's post-installation documentation.

2. Install ROCm and verify the GPU

Install a compatible ROCm release

For the earlier ROCm 7.1.1-based environment, the installer package for Ubuntu 24.04 was:

cd ~/Downloads wget https://repo.radeon.com/amdgpu-install/7.1.1/ubuntu/noble/amdgpu-install_7.1.1.70101-1_all.deb sudo apt install ./amdgpu-install_7.1.1.70101-1_all.deb

The Ryzen installation procedure used the Ubuntu inbox kernel driver rather than installing a separate DKMS driver:

sudo amdgpu-install -y --usecase=rocm --no-dkms sudo usermod -aG render,video "$USER" sudo reboot

Version-specific note: These commands document the earlier ROCm 7.1.1 installation path; they are not a recommendation to downgrade a currently working ROCm installation. AMD's ROCm 7.1.1 Ryzen installation guide specifies additional OS and kernel requirements. New installations should use the instructions for their chosen, supported release.

Check detection after rebooting

On the Ubuntu host, run:

uname -r rocminfo | grep -E 'Name:|gfx1151' rocm-smi

If available, rocminfo should identify a GPU agent with the target gfx1151. In my recorded output, ROCm detected the Radeon 8060S / Ryzen AI Max+ 395 and reported the expected gfx1151 architecture. A GPU memory figure of approximately 96 GiB also appeared in one of my detection logs; on this APU, that figure describes GPU-accessible memory in a unified-memory system, not a separate 96 GiB graphics card.

The availability and output format of rocm-smi depend on the ROCm package version. Some newer tooling also uses amd-smi.

3. Run Ollama and Open WebUI with Docker Compose

I used separate containers for Ollama and its web interface, with persistent volumes for downloaded models and application data.

mkdir -p ~/ollama-rocm cd ~/ollama-rocm nano docker-compose.yml

Here is a locally bound adaptation of the original Compose configuration. The original notes published ports on all host interfaces; binding them to 127.0.0.1 is a safer starting point when the browser runs on the same computer.

services: ollama: image: ollama/ollama:rocm container_name: ollama restart: unless-stopped environment: OLLAMA_HOST: 0.0.0.0:11434 devices: - /dev/kfd:/dev/kfd - /dev/dri:/dev/dri volumes: - ollama:/root/.ollama ports: - "127.0.0.1:11434:11434" open-webui: image: ghcr.io/open-webui/open-webui:main container_name: open-webui restart: unless-stopped environment: OLLAMA_BASE_URL: http://ollama:11434 ports: - "127.0.0.1:3000:8080" volumes: - open-webui:/app/backend/data depends_on: - ollama volumes: ollama: open-webui:

The key settings are:

/dev/kfd and /dev/dri expose the AMD GPU interfaces to the Ollama container.
OLLAMA_HOST=0.0.0.0:11434 lets Ollama listen inside its container so Open WebUI can reach it over the Compose network. This is different from publishing the port on every host interface.
OLLAMA_BASE_URL=http://ollama:11434 uses the Compose service name for container-to-container communication.
Named volumes preserve models and Open WebUI data when containers are recreated.

About `HSA_OVERRIDE_GFX_VERSION`

My original configuration included:

environment: - HSA_OVERRIDE_GFX_VERSION=11.5.1 - OLLAMA_HOST=0.0.0.0:11434

This was a compatibility override used in the earlier Strix Halo setup. It should not be treated as a universally required setting. Current Ollama AMD GPU documentation explicitly lists the Ryzen AI Max+ 395 and gfx1151. Start without the override on a supported stack; consider an override only when diagnosing a specific compatibility problem with a known software version.

Start the services

cd ~/ollama-rocm docker compose config docker compose up -d docker compose ps docker logs --tail 50 ollama

docker compose config checks and renders the Compose configuration. A running container and an Ollama listening message indicate that the server started, not that GPU inference has been verified.

Open http://localhost:3000 to access Open WebUI and configure its initial account.

Security and reproducibility: ollama/ollama:rocm and ghcr.io/open-webui/open-webui:main are moving image tags. For a reproducible deployment, pin tested image versions or digests and record them alongside the host kernel and ROCm versions. If you want to access Open WebUI from another machine, configure authentication and appropriate network restrictions before exposing its port. Avoid exposing the Ollama API directly to untrusted networks.

4. Download and run a model

The model names in my original notes were:

docker exec ollama ollama pull llama3.3:70b docker exec ollama ollama pull qwen3-coder:30b docker exec ollama ollama pull gemma3:27b-it-qat

These are alternatives; there is no need to download all three merely to verify GPU access. For a single model, start an interactive session:

docker exec -it ollama ollama run llama3.3:70b

The exact tag matters. llama3.3 currently resolves to a 70B model, but explicitly using llama3.3:70b keeps the commands consistent with the model you downloaded.

Other useful commands are:

docker exec ollama ollama list docker exec ollama ollama ps # Remove a model only when it is no longer needed: # docker exec ollama ollama rm llama3.3:70b

Large models can require substantial system memory and GPU-addressable memory, especially with longer context windows. Model size alone does not guarantee that inference will fit or remain stable.

5. Verify that inference is actually using the GPU

Successful model execution is only the first check. I used two complementary methods.

Check Ollama's processor allocation

While the model is loaded, run:

docker exec ollama ollama ps

Inspect the PROCESSOR column. It indicates whether Ollama has placed the model on GPU, CPU, or a mixture of both. This is more informative than assuming GPU acceleration because a ROCm-enabled image was used.

Monitor GPU activity on the host

In another terminal:

watch -n 1 rocm-smi

Run a prompt while monitoring utilization and memory reporting. A change in GPU activity can support the conclusion that the GPU is being used, but the readings may vary across tools and UMA devices. Use the Ollama allocation report together with the host metrics and server logs.

My recorded logs confirmed that Ollama discovered a ROCm device identified as gfx1151. Subsequent checks also included successful inference requests. These observations establish GPU detection and operation in those tested configurations; I did not record a controlled tokens-per-second benchmark for this article.

6. Troubleshooting: GPU detected, but model execution fails

A particularly instructive problem appeared during later maintenance. After an Ubuntu kernel update, ROCm still detected gfx1151, but Ollama could fail during inference, including runner errors and an out-of-memory error. In the recorded investigation, kernel 6.17.0-19-generic was associated with failure, while a previously working 6.17.0-14-generic configuration helped isolate the regression.

This is a case-specific observation, not evidence that every newer kernel is incompatible. AMD's supported kernel requirements have changed since then, including fixes for GPU queue creation and memory checks on Ryzen AI Max devices. Always follow the kernel requirements for your installed ROCm stack.

For troubleshooting, collect these diagnostics before changing drivers or kernel settings:

uname -r rocminfo | grep -E 'Name:|gfx1151' docker exec ollama ollama --version docker exec ollama ollama ps docker logs --tail 150 ollama ls -l /dev/kfd /dev/dri/

Check the results in this order:

Device detection: Does rocminfo list the expected gfx1151 GPU?
Container access: Are /dev/kfd and /dev/dri passed through, and are permissions sufficient?
Runtime compatibility: Are the host kernel, ROCm-related driver components, and Ollama's bundled ROCm libraries compatible?
Memory allocation: Does the failure occur only when a model or large context is loaded?
Regression isolation: Did the problem begin after a specific package or kernel update?

An out-of-memory message is not equivalent to “GPU not detected.” Likewise, a healthy rocminfo result does not prove that a model runner can allocate memory successfully. Avoid applying overrides or downgrading kernels without first identifying the cause.

For more on device permissions and logs, see Ollama troubleshooting and AMD's RDNA 3.5 kernel notes.

7. Limitations and lessons learned

The practical workflow is straightforward once the stack is compatible: check the AMD GPU on the host, pass its device nodes into an Ollama ROCm container, run a model, and verify GPU allocation with ollama ps and host monitoring.

The difficult part is version compatibility. The CPU, integrated GPU, and memory capacity do not by themselves establish that an arbitrary Ubuntu kernel, ROCm release, or Ollama container will work together. My later kernel regression reinforced the importance of recording the exact versions of the components that were actually tested.

For a reproducible setup, I recommend recording uname -r, the ROCm version, image digests, model tags, and whether inference was confirmed on GPU. The commands above are based on my documented setup and official references, but the revised example has not been rerun end-to-end as a fresh installation for this article.

References

AI disclosure: This article documents my own technical setup and results. An LLM was used to help draft, organize, and edit the English text. I remain responsible for its technical accuracy.

原文：https://www.a8q9.cn/en/article/running-ollama-on-amd-ryzen-ai-max-395-with-rocm-and-docker-421621.html