Skip to content

Commit 94bef5f

Browse files
committed
Add section to readme on container images
Signed-off-by: Samuel Monson <[email protected]>
1 parent 147cd50 commit 94bef5f

File tree

1 file changed

+20
-0
lines changed

1 file changed

+20
-0
lines changed

README.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,26 @@ pip install git+https://github.com/vllm-project/guidellm.git
5252

5353
For detailed installation instructions and requirements, see the [Installation Guide](https://github.com/vllm-project/guidellm/blob/main/docs/install.md).
5454

55+
### With Podman / Docker
56+
57+
Alternatively we publish container images at [ghcr.io/vllm-project/guidellm](https://github.com/vllm-project/guidellm/pkgs/container/guidellm). Running a container is (by default) equivalent to `guidellm benchmark run`:
58+
59+
``` bash
60+
podman run \
61+
--rm -it \
62+
-v "./results:/results:rw" \
63+
-e GUIDELLM_TARGET=http://localhost:8000 \
64+
-e GUIDELLM_RATE_TYPE=sweep \
65+
-e GUIDELLM_MAX_SECONDS=30 \
66+
-e GUIDELLM_DATA="prompt_tokens=256,output_tokens=128" \
67+
ghcr.io/vllm-project/guidellm:latest
68+
```
69+
70+
> [!TIP]
71+
> CLI options can also be specified as ENV variables (E.g. `--rate-type sweep` -> `GUIDELLM_RATE_TYPE=sweep`). If both are specified then the CLI option overrides the the ENV.
72+
73+
Replace `latest` with `stable` for the newest tagged release or set a specific release if desired.
74+
5575
### Quick Start
5676

5777
#### 1. Start an OpenAI Compatible Server (vLLM)

0 commit comments

Comments
 (0)