Thanks for your reply. A practical follow up: do you mount storage from the host or do you use virtual drives?

I use ZFS on my host and and file recovery is just so convenient when the services can write directly to it. I’m reluctant to give that up but assume a mount reduces isolation.

Title made me suspicious already.

As an alternative to mounting full /dev/dri, there are symlinks under /dev/dri/by-path/ which you can use. IIUC they are linked by the used PCI slot, hence should be as stable as the physical order of the cards.

I use this pattern in a compose file:

...
devices:
  - source: /dev/dri/by-path/pci-XXX-render
    target: /dev/dri/renderD128
    permissions: rwm

In case you are unaware, friends don’t let friends use ollama. The llama.cpp CLI has become pretty much equally easy to use, plus since you use Docker there’s no need to compile anything.

Edit: typos and missing words

Haha, that sounds like procrastination to me.

There are tools like crontab.guru which helped me get started.