Here to show my experience about playing with ROCm with runable code, step-by-step tutorial to help you reproduce what I have did. If you have iGPU or dGPU of AMD, you may try Machine Learning with them.
NOTICE : For more easier tracking my update, I use 🆕 and 🔥 to flag the new hot topics.
- Deploy Deepseek-R1 in one GPU -AMD Instinct™ MI300X 🔥
- Deploy Llama 3.2 Vision quickly on AMD ROCm with Ollama
- Deploy vLLM service with Kubernetes over AMD ROCm GPU
- Deploy LLM with Radeon iGPU 780M
- Examples of using vLLM with ROCm 🆕 🔥
- Help scripts to fast use vLLM with ROCm
- Example: using docker compose to run multiple containers of vllm serve. e.g. gpu=0,1 for container-1 and gpu=6,7 for container-2.
- vLLM
- Neural Magic vLLM, nm-vllm
- AIBrix
- KubeAI : AI Inferencing Operator
- vLLM Production Stack
- RAG_LLM_QnA_Assistant, Step-by-step tutorial repo project to setup RAG Apps with ROCm
- Ask4ROCm_Chatbot, An chatbot app drive by RAG solution.
- LLM_Voice_Assistant , Use STT/TTS model from Picovoice.
- Easy-Wav2Lip-ROCm, Easy run Wav2Lip with ROCm over AMD GPU. Way2Lip is a project of Generalized Lip Sync Models
- Run EchoMimic with ROCm EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditioning
- Run LLama-3.2-vision with ROCm Ollama+Llama-3.2-vision+ROCm
- Deploy vLLM service with Kubernetes over AMD ROCm GPU , Turoial with sample codes.
- Play Qwen2.5–Omni with AMD GPU 🆕 🔥
These projects may not offical announce to support ROCm GPU. But they work fine base on my verification.
| Name | URL | Category | Hands on |
|---|---|---|---|
| CLM-4-Voice | https://github.com/THUDM/GLM-4-Voice | Conversation AI | |
| EchoMimic | https://github.com/BadToBest/EchoMimic | Digital Human GenAI | Run EchoMimic with ROCm |
| Easy-Wav2Lip | https://github.com/anothermartz/Easy-Wav2Lip | Digital Human GenAI | Easy-Wav2Lip-ROCm |
| GOT-OCR2 | https://github.com/Ucas-HaoranWei/GOT-OCR2.0 | end2end OCR | |
| Moshi | https://github.com/kyutai-labs/moshi | Conversation AI | |
| mini-omni | https://github.com/gpt-omni/mini-omni | Conversation AI | |
| mini-omni2 | https://github.com/gpt-omni/mini-omni2 | Conversation AI | |
| Picovoice/orca | https://github.com/Picovoice/orca | Conversation AI | LLM_Voice_Assistant |
| Retrieval-based-Voice-Conversion-WebUI | https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI.git | Easily train a good VC model with voice data <= 10 mins! | |
| Freeze-Omni 🆕 🔥 | https://github.com/VITA-MLLM/Freeze-Omni | A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLM | Realtime on Radeon W7900, realtime with good response, feel good than Moshi, mini-omni2 |
| Step-Auido 🆕 🔥 | https://github.com/stepfun-ai/Step-Audio | Convseration AI | Too big model, not real time |
| Step-Video-T2V 🆕 🔥 | https://github.com/stepfun-ai/Step-Video-T2V | Video GenAI | Run with 1xMI300X |
| UI-TARS | https://github.com/bytedance/UI-TARS | Automated GUI Interaction with Native Agentsfrom ByteDance | |
| Qwen2.5-Omni 🆕 🔥 | https://github.com/QwenLM/Qwen2.5-Omni | end-to-end multimodal model in the Qwen serie | |
| CosyVoice | https://github.com/FunAudioLLM/CosyVoice | TTS LLM | tutorial , |
- Tutorial: vLLM deploy
- Summary: Awesome-Agent-Framework - https://github.com/AgentSpaceAI/Awesome-Agent-Framework - https://github.com/kyrolabs/awesome-agents
@misc{ Playing with ROCm,
author = {He Ye (Alex)},
title = {Playing with ROCm: share my experience and practice},
howpublished = {\url{https://alexhegit.github.io/}},
year = {2024--}
}