Skip to content

Commit fb58e3a

Browse files
authored
[Docs] Update docker.md with HF_TOKEN, new model, and podman fix (#21856)
1 parent 76080cf commit fb58e3a

File tree

1 file changed

+5
-5
lines changed

1 file changed

+5
-5
lines changed

docs/deployment/docker.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,23 +10,23 @@ The image can be used to run OpenAI compatible server and is available on Docker
1010
```bash
1111
docker run --runtime nvidia --gpus all \
1212
-v ~/.cache/huggingface:/root/.cache/huggingface \
13-
--env "HUGGING_FACE_HUB_TOKEN=<secret>" \
13+
--env "HUGGING_FACE_HUB_TOKEN=$HF_TOKEN" \
1414
-p 8000:8000 \
1515
--ipc=host \
1616
vllm/vllm-openai:latest \
17-
--model mistralai/Mistral-7B-v0.1
17+
--model Qwen/Qwen3-0.6B
1818
```
1919

2020
This image can also be used with other container engines such as [Podman](https://podman.io/).
2121

2222
```bash
23-
podman run --gpus all \
23+
podman run --device nvidia.com/gpu=all \
2424
-v ~/.cache/huggingface:/root/.cache/huggingface \
2525
--env "HUGGING_FACE_HUB_TOKEN=$HF_TOKEN" \
2626
-p 8000:8000 \
2727
--ipc=host \
28-
vllm/vllm-openai:latest \
29-
--model mistralai/Mistral-7B-v0.1
28+
docker.io/vllm/vllm-openai:latest \
29+
--model Qwen/Qwen3-0.6B
3030
```
3131

3232
You can add any other [engine-args](../configuration/engine_args.md) you need after the image tag (`vllm/vllm-openai:latest`).

0 commit comments

Comments
 (0)