docs(verl): add verl training setup and launch instructions#73
Conversation
| uv pip install --no-deps -e verl/ --torch-backend=cu130 | ||
|
|
||
| # Download and install the stable branch vllm (support vLLM serving in RL rollout) of agentcore-rl-toolkit | ||
| git clone https://github.com/awslabs/agentcore-rl-toolkit -b vllm |
There was a problem hiding this comment.
do we need the vllm branch?
There was a problem hiding this comment.
Don't need it, will remove.
| uv pip install -e rllm/rllm-model-gateway | ||
| ``` | ||
|
|
||
| :::note |
There was a problem hiding this comment.
maybe add this at the top so people don't install the wrong version but only realize until the end?
| uv venv --python 3.12 --seed | ||
| source .venv/bin/activate | ||
|
|
||
| uv pip install vllm==0.21.0 --torch-backend=cu130 |
There was a problem hiding this comment.
looks like v0.8.0 only verified vllm==0.20.2? https://github.com/verl-project/verl/releases/tag/v0.8.0. should we be more conservative or is there some features that we need post 0.20.2?
There was a problem hiding this comment.
My test is using vllm 0.21.0 when verl 0.8.0 is not released yet. But yeah using the official recommended vllm version is good. I will change.
| ## Prerequisites | ||
|
|
||
| - A GPU cluster with **CUDA>=12.8** installed. | ||
| - Python 3.10+ and [`uv`](https://docs.astral.sh/uv/). |
There was a problem hiding this comment.
@lyzustc should we raise to python 3.12? I just realized yesterday megatron-bridge (post v0.4.2) that's compatible with verl 0.8.0 actually already dropped python 3.10 support (minimum python 3.12). see release notes on the github page here https://github.com/NVIDIA-NeMo/Megatron-Bridge
There was a problem hiding this comment.
Good catch, will change it.
…ject.toml and megatron commands
3 participants
Add verl training setup and launch instructions to official docs.
See the rendered doc page at https://lyzustc.github.io/agentcore-rl-toolkit/guides/verl-backend-setup/.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.