Create a Ubuntu 24.04 LXC container.
I used the excellent tteck script but you can also do using any other method you are comfortable with.
Give it plenty of specs regarding storage, RAM and CPU (according to Ollama's recommendations)
I chose 32GB and all available cores. Give it plenty of storage to accomodate your models (can be expanded using the GUI)
I chose 60 GB to start. ROCm takes up ~30 GB.
In the LXC container:
install ROCm according to official instructions but...
Read more
I used the excellent tteck script but you can also do using any other method you are comfortable with.
Give it plenty of specs regarding storage, RAM and CPU (according to Ollama's recommendations)
I chose 32GB and all available cores. Give it plenty of storage to accomodate your models (can be expanded using the GUI)
I chose 60 GB to start. ROCm takes up ~30 GB.
In the LXC container:
install ROCm according to official instructions but...
Read more