"build-vllm-docker.py" file doesn't work

[Chr1st1anAI]

I'm running this command (REPO: https://github.com/ROCm/tritoninferenceserver-vllm)
python3 ~/tritoninferenceserver-vllm/build-vllm-docker.py --no-container-pull --enable-rocm --endpoint=grpc --image gpu-base,rocm/pytorch:rocm6.4_ubuntu22.04_py3.10_pytorch_release_2.6.0 --endpoint=http --backend=python --backend=vllm

I've been making changes to the "build-vllm-docker.py" file. I managed to compile the default "tritonserver" by making some changes, but it's quite outdated and doesn't allow vLLM to deploy any models.
I was trying to compile Triton Inference Server with the image "rocm/pytorch:rocm6.4_ubuntu24.04_py3.12_pytorch_release_2.6.0" but couldn't make progress. I'm currently testing with the image "rocm/pytorch:rocm6.4_ubuntu22.04_py3.10_pytorch_release_2.6.0", which is progressing further. As errors arise, I continue to resolve them...

The error I have now is:
95.46 /var/lib/jenkins/libs/flash-attention/csrc/flash_attn_rocm/composable_kernel/include/ck/tensor_operation/gpu/block/blockwise_gemm_xdlops_hip.hpp:870:32: error: no member named 'a_origin' in 'BlockwiseGemmXdlops_v2<BlockSize, FloatAB, FloatAcc, ATileDesc, BTileDesc, AMmaTileDesc, BMmaTileDesc, MPerBlock, NPerBlock, KPerBlock, MPerXDL, NPerXDL, MRepeat, NRepeat, KPack, TransposeC, AMmaKStride, BMmaKStride>'
95.46 870 | : a_thread_copy_(other.a_origin), b_thread_copy_(other.b_origin)
95.46 | ~~~~~ ^
95.46 /var/lib/jenkins/libs/flash-attention/csrc/flash_attn_rocm/composable_kernel/include/ck/tensor_operation/gpu/block/blockwise_gemm_xdlops_hip.hpp:870:64: error: no member named 'b_origin' in 'BlockwiseGemmXdlops_v2<BlockSize, FloatAB, FloatAcc, ATileDesc, BTileDesc, AMmaTileDesc, BMmaTileDesc, MPerBlock, NPerBlock, KPerBlock, MPerXDL, NPerXDL, MRepeat, NRepeat, KPack, TransposeC, AMmaKStride, BMmaKStride>'
95.46 870 | : a_thread_copy_(other.a_origin), b_thread_copy_(other.b_origin)
95.46 | ~~~~~ ^

NOTE: This would be the ideal image to compile
"rocm/pytorch:rocm6.4_ubuntu24.04_py3.12_pytorch_release_2.6.0"

[ffloresy]

Docker Image can be built by updating file build-vllm-docker.py with:

Replace line 1078:
https://boostorg.jfrog.io/artifactory/main/release/1.80.0/source/boost_1_80_0.tar.gz && \
with:
https://archives.boost.io/release/1.80.0/source/boost_1_80_0.tar.gz && \

Add between line 334 and 335:
self.cmd("sed -i 's|URL https://boostorg.jfrog.io/artifactory/main/release/1.79.0/source/boost_1_79_0.tar.gz|URL https://archives.boost.io/release/1.79.0/source/boost_1_79_0.tar.gz|' /tmp/tritonbuild/python/CMakeLists.txt")

[yaoliu13]

I would like to collect some information from you to help us prioritize this.

1. What specific AMD GPU are you using?
2. What company or university are you working for?

[Chr1st1anAI]

• What specific AMD GPU are you using?
• What company or university are you working for?

Hi,

1. MI250x
2. Saptiva INC.

NOTE: This would be the ideal image to compile
rocm6.4.1_ubuntu24.04_py3.12_pytorch_release_2.7.1

[yaoliu13]

Thank you. We will have a release here: https://rocm.docs.amd.com/en/latest/how-to/deep-learning-rocm.html

Meanwhile, we also have

1. https://rocm.blogs.amd.com/search.html?q=vllm
2. https://rocm.blogs.amd.com/search.html?q=sglang
3. HIP: Enable Matrix cores for MMQ Kernels, Enable stream-K for CDNA 3 ggml-org/llama.cpp#14624