Skip to content

Improve README to match Appverse documentation standard #1

Open
Open
Improve README to match Appverse documentation standard#1

Description

@a-pasquale
a-pasquale
opened on Feb 24, 2026

Hi! As part of the OOD Appverse community, we're working to improve documentation consistency across Open OnDemand apps so that deployers at other sites can more easily evaluate, install, and adapt them.

We've put together a README template that covers the key sections deployers typically need when considering an app for their site.

After reviewing your current README, here's what we found:

Sections to add (not currently in your README):

  • Overview (expanded description with app type, upstream link, and technical summary)
  • Screenshots
  • Features
  • Requirements (compute node software, OOD version)
  • App Installation (OOD-specific clone and configure steps)
  • Configuration (form.yml.erb attributes table)
  • Troubleshooting
  • Testing
  • Known Limitations
  • Contributing
  • References
  • Acknowledgments

Sections that could be expanded:

  • Title and description -- could mention the Batch Connect basic template, Slurm scheduler, and link to the upstream Gradio and llama.cpp projects
  • Obtaining an LLM -- useful content that could be preserved in a Software Installation subsection
  • Installing the Python environment -- useful content that could be preserved in the installation steps

Sections already present:

  • Title -- clear app name
  • Contact information -- maintainer name, email, and script author credit
  • Brief description -- identifies the app, the model, and the DEAC cluster
  • Obtaining an LLM -- download instructions for the GGUF model
  • Installing the Python environment -- venv setup and pip install instructions

Below we've provided two versions: a diff showing exactly what we're suggesting to add or change, and a clean copy-paste version you can drop in directly. Lines marked with <!-- TODO --> need your input -- we deliberately left those rather than guessing.

Diff view -- see exactly what's new and changed 
  # Gradio Chatbot App for Open OnDemand (OOD)

- Wake Forest University (WFU) HPC Team<br>
- Primary Contact: Sean Anderson (anderss@wfu.edu)<br>
- `chatbot.py` script created by Cody Stevens (stevca9@wfu.edu).
-
- This is a quick way to get a simple LLM chatbot app in OOD. This is known to
- work with the Meta Llama-3.1-8B-Instruct model and using Gradio.app as a
- frontend interface for interacting with it.
-
-
- ## Obtaining an LLM
-
- Make a directory for your LLM:
-
- ```sh
- mkdir -p ${HOME}/llm
- ```
-
- For this example, we'll use a pre-processed model from HuggingFace that can be
- downloaded directly in the GGUF format:
-
- ```sh
- wget https://huggingface.co/professorf/Meta-Llama-3-1-8B-Instruct-f16-gguf/resolve/main/llama-3-1-8b-instruct-f16.gguf -o ${HOME}/llm/llama-3-1-8b-instruct-f16.gguf
- ```
-
- This model is around 16 GB in size.
-
-
- ## Installing the Python environment
-
- Now create a fresh Python environment using any version of Python 3 that you
- want. Activate the environment after creation:
-
- ```sh
- python3 -m venv ${HOME}/env-chatbot # create environment
- . ${HOME}/env-chatbot/bin/activate  # activate environment
- ```
-
- Lastly, install the required modules into the environment:
-
- ```sh
- python3 -m pip install llama_cpp_python gradio openai matplotlib
- ```
+ ## Overview
+
+ An [Open OnDemand](https://openondemand.org/) Batch Connect app that launches
+ a [Gradio](https://www.gradio.app/) LLM chatbot as an interactive web server
+ session on HPC clusters. The app uses
+ [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) to run Meta's
+ Llama-3.1-8B-Instruct model in GGUF format and provides a chat interface
+ through a Gradio web UI.
+
+ This app uses the Batch Connect `basic` template to run a Gradio web server
+ on a compute node. The `before.sh.erb` script dynamically generates
+ `chatbot.py` at launch time with the appropriate server configuration.
+
+ Developed by the Wake Forest University (WFU) HPC Team.
+ Primary Contact: Sean Anderson (anderss@wfu.edu)
+ `chatbot.py` script created by Cody Stevens (stevca9@wfu.edu)
+
+ - **Upstream project:** [Gradio](https://www.gradio.app/) + [llama-cpp-python](https://github.com/abetlen/llama-cpp-python)
+ - **Batch Connect template:** `basic`
+ - **Scheduler:** Slurm
+ - **Cluster:** DEAC
+
+ ## Screenshots
+
+ <!-- TODO: Add a screenshot of the Gradio chatbot interface in the browser -->
+
+ ## Features
+
+ - Launches a Gradio LLM chatbot web server on compute nodes via Batch
+   Connect `basic` template
+ - Chat with Meta Llama-3.1-8B-Instruct model through a web browser
+ - Dark theme enabled by default
+ - Streaming responses with conversation history
+ - GPU-accelerated inference via llama-cpp-python (`n_gpu_layers=-1`)
+ - Configurable Slurm account, partition, node type, cores, memory, and GPUs
+   via the launch form
+ - Working directory selector
+ - Optional additional environment modules can be loaded at launch time
+ - Multiple CPU and GPU node types supported (AMD Zen3/Zen4, Intel Cascade
+   Lake/Skylake, A100, V100)
+ - Module loaded at runtime: `envs/llama-server`
+
+ ## Requirements
+
+ ### Compute Node Software
+
+ - LLM server Python environment available as an environment module
+   (currently `envs/llama-server`)
+ - [Environment Modules](https://modules.readthedocs.io/) with init at
+   `/usr/share/Modules/init/bash`
+ - Python 3 with `llama-cpp-python`, `gradio`, `openai`, and `matplotlib`
+ - NVIDIA CUDA compatibility libraries (the script sets `LD_LIBRARY_PATH`
+   for CUDA 12.4 compat)
+ - A GGUF-format LLM model file accessible from compute nodes
+ - OpenSSL (used for password generation)
+
+ ### Open OnDemand
+
+ <!-- TODO: Specify the minimum OOD version this app has been tested with -->
+ - Slurm scheduler
+
+ ## App Installation
+
+ ### 1. Clone the repository
+
+ ```sh
+ cd /var/www/ood/apps/sys
+ git clone https://github.com/WFU-HPC/OOD-apps.server.gradio-chatbot.git
+ cd OOD-apps.server.gradio-chatbot
+ ```
+
+ ### 2. Obtain an LLM model
+
+ Download a GGUF-format model. For example, the Meta Llama-3.1-8B-Instruct
+ model (~16 GB):
+
+ ```sh
+ mkdir -p ${HOME}/llm
+ wget https://huggingface.co/professorf/Meta-Llama-3-1-8B-Instruct-f16-gguf/resolve/main/llama-3-1-8b-instruct-f16.gguf -o ${HOME}/llm/llama-3-1-8b-instruct-f16.gguf
+ ```
+
+ ### 3. Set up the Python environment
+
+ Create a Python environment with the required packages:
+
+ ```sh
+ python3 -m venv ${HOME}/env-chatbot
+ . ${HOME}/env-chatbot/bin/activate
+ python3 -m pip install llama_cpp_python gradio openai matplotlib
+ ```
+
+ Make this environment available as an environment module (e.g.,
+ `envs/llama-server`), or update `script.sh.erb` to activate it directly.
+
+ ### 4. Configure for your site
+
+ Edit `form.yml.erb` and update these values for your cluster:
+
+ | Attribute   | WFU Default            | Change to                          |
+ |-------------|------------------------|------------------------------------|
+ | `cluster`   | `"deac"`               | Your cluster name                  |
+ | `accounts`  | WFU Slurm accounts     | Your Slurm accounts               |
+ | `queues`    | `"ondemand"`           | Your Slurm partition               |
+ | `node_type` | WFU-specific node types| Node types on your cluster         |
+
+ In `before.sh.erb`, update the `model_path` to point to your GGUF model
+ file location. The WFU default is:
+ ```
+ /deac/data/models/llama-cpp/llama-3_1-8b-instruct-f16.gguf
+ ```
+
+ In `script.sh.erb`, update the `LD_LIBRARY_PATH` to match your CUDA
+ installation and ensure the `envs/llama-server` module is available:
+ ```
+ module load envs/llama-server
+ ```
+
+ ### 5. Verify
+
+ No OOD restart is needed (Batch Connect apps are detected automatically).
+ Visit your OOD dashboard and look for **Gradio Chatbot** under
+ **Interactive Apps > Machine Learning and LLMs**.
+
+ ## Configuration
+
+ ### form.yml.erb attributes
+
+ | Attribute              | Widget          | Description                                   | Default                    |
+ |------------------------|-----------------|-----------------------------------------------|----------------------------|
+ | `enable_extra_modules` | check_box       | Toggle extra module loading                   | unchecked                  |
+ | `extra_modules`        | text_field      | Additional environment modules to load        | `""`                       |
+ | `working_dir`          | path_selector   | Working directory for the session              | User's `$HOME`             |
+ | `accounts`             | select          | Slurm account for job submission               | (dynamic from cluster)     |
+ | `queues`               | select          | Slurm partition                                | `"ondemand"`               |
+ | `bc_num_hours`         | number          | Maximum wall time (hours)                      | `1`                        |
+ | `num_cores`            | number_field    | Number of CPU cores                            | `1`                        |
+ | `memory`               | number_field    | Memory in GB                                   | `8`                        |
+ | `node_type`            | select          | Compute node type (CPU, GPU, specific HW)      | Any (shortest wait)        |
+ | `num_gpus`             | number_field    | Number of GPUs (for GPU node types)             | `0`                        |
+
+ ## Troubleshooting
+
+ <!-- TODO: Add troubleshooting tips -->
+
+ ## Testing
+
+ <!-- TODO: Update with sites where this app has been deployed -->
+
+ | Site                          | OOD Version   | Scheduler | Status |
+ |-------------------------------|---------------|-----------|--------|
+ | Wake Forest University (DEAC) | <!-- TODO --> | Slurm     | <!-- TODO --> |
+
+ ## Known Limitations
+
+ <!-- TODO: Document any known limitations -->
+
+ ## Contributing
+
+ Contributions are welcome. To contribute:
+
+ 1. Fork this repository
+ 2. Create a feature branch (`git checkout -b feature/my-improvement`)
+ 3. Submit a pull request with a description of your changes
+
+ For bugs or feature requests,
+ [open an issue](https://github.com/WFU-HPC/OOD-apps.server.gradio-chatbot/issues).
+
+ ## References
+
+ - [Gradio](https://www.gradio.app/) -- the web UI framework used for the
+   chat interface
+ - [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) -- Python
+   bindings for llama.cpp, used for LLM inference
+ - [Meta Llama 3.1](https://ai.meta.com/llama/) -- the LLM family this app
+   is configured to use
+ - [Open OnDemand](https://openondemand.org/) -- the HPC portal framework
+ - [OOD Batch Connect app development docs](https://osc.github.io/ood-documentation/latest/app-development.html)
+
+ ### Software Installation
+
+ <!-- TODO: Add notes about building the llama-server environment module on your compute nodes -->
+
+ ## License
+
+ [MIT License](LICENSE)
+
+ ## Acknowledgments
+
+ <!-- TODO: Add funding or institutional support information -->
Clean README.md -- copy-paste ready 
# Gradio Chatbot App for Open OnDemand (OOD)

## Overview

An [Open OnDemand](https://openondemand.org/) Batch Connect app that launches
a [Gradio](https://www.gradio.app/) LLM chatbot as an interactive web server
session on HPC clusters. The app uses
[llama-cpp-python](https://github.com/abetlen/llama-cpp-python) to run Meta's
Llama-3.1-8B-Instruct model in GGUF format and provides a chat interface
through a Gradio web UI.

This app uses the Batch Connect `basic` template to run a Gradio web server
on a compute node. The `before.sh.erb` script dynamically generates
`chatbot.py` at launch time with the appropriate server configuration.

Developed by the Wake Forest University (WFU) HPC Team.
Primary Contact: Sean Anderson (anderss@wfu.edu)
`chatbot.py` script created by Cody Stevens (stevca9@wfu.edu)

- **Upstream project:** [Gradio](https://www.gradio.app/) + [llama-cpp-python](https://github.com/abetlen/llama-cpp-python)
- **Batch Connect template:** `basic`
- **Scheduler:** Slurm
- **Cluster:** DEAC

## Screenshots

<!-- TODO: Add a screenshot of the Gradio chatbot interface in the browser -->

## Features

- Launches a Gradio LLM chatbot web server on compute nodes via Batch
  Connect `basic` template
- Chat with Meta Llama-3.1-8B-Instruct model through a web browser
- Dark theme enabled by default
- Streaming responses with conversation history
- GPU-accelerated inference via llama-cpp-python (`n_gpu_layers=-1`)
- Configurable Slurm account, partition, node type, cores, memory, and GPUs
  via the launch form
- Working directory selector
- Optional additional environment modules can be loaded at launch time
- Multiple CPU and GPU node types supported (AMD Zen3/Zen4, Intel Cascade
  Lake/Skylake, A100, V100)
- Module loaded at runtime: `envs/llama-server`

## Requirements

### Compute Node Software

- LLM server Python environment available as an environment module
  (currently `envs/llama-server`)
- [Environment Modules](https://modules.readthedocs.io/) with init at
  `/usr/share/Modules/init/bash`
- Python 3 with `llama-cpp-python`, `gradio`, `openai`, and `matplotlib`
- NVIDIA CUDA compatibility libraries (the script sets `LD_LIBRARY_PATH`
  for CUDA 12.4 compat)
- A GGUF-format LLM model file accessible from compute nodes
- OpenSSL (used for password generation)

### Open OnDemand

<!-- TODO: Specify the minimum OOD version this app has been tested with -->
- Slurm scheduler

## App Installation

### 1. Clone the repository

```sh
cd /var/www/ood/apps/sys
git clone https://github.com/WFU-HPC/OOD-apps.server.gradio-chatbot.git
cd OOD-apps.server.gradio-chatbot
```

### 2. Obtain an LLM model

Download a GGUF-format model. For example, the Meta Llama-3.1-8B-Instruct
model (~16 GB):

```sh
mkdir -p ${HOME}/llm
wget https://huggingface.co/professorf/Meta-Llama-3-1-8B-Instruct-f16-gguf/resolve/main/llama-3-1-8b-instruct-f16.gguf -o ${HOME}/llm/llama-3-1-8b-instruct-f16.gguf
```

### 3. Set up the Python environment

Create a Python environment with the required packages:

```sh
python3 -m venv ${HOME}/env-chatbot
. ${HOME}/env-chatbot/bin/activate
python3 -m pip install llama_cpp_python gradio openai matplotlib
```

Make this environment available as an environment module (e.g.,
`envs/llama-server`), or update `script.sh.erb` to activate it directly.

### 4. Configure for your site

Edit `form.yml.erb` and update these values for your cluster:

| Attribute   | WFU Default            | Change to                          |
|-------------|------------------------|------------------------------------|
| `cluster`   | `"deac"`               | Your cluster name                  |
| `accounts`  | WFU Slurm accounts     | Your Slurm accounts               |
| `queues`    | `"ondemand"`           | Your Slurm partition               |
| `node_type` | WFU-specific node types| Node types on your cluster         |

In `before.sh.erb`, update the `model_path` to point to your GGUF model
file location. The WFU default is:
```
/deac/data/models/llama-cpp/llama-3_1-8b-instruct-f16.gguf
```

In `script.sh.erb`, update the `LD_LIBRARY_PATH` to match your CUDA
installation and ensure the `envs/llama-server` module is available:
```
module load envs/llama-server
```

### 5. Verify

No OOD restart is needed (Batch Connect apps are detected automatically).
Visit your OOD dashboard and look for **Gradio Chatbot** under
**Interactive Apps > Machine Learning and LLMs**.

## Configuration

### form.yml.erb attributes

| Attribute              | Widget          | Description                                   | Default                    |
|------------------------|-----------------|-----------------------------------------------|----------------------------|
| `enable_extra_modules` | check_box       | Toggle extra module loading                   | unchecked                  |
| `extra_modules`        | text_field      | Additional environment modules to load        | `""`                       |
| `working_dir`          | path_selector   | Working directory for the session              | User's `$HOME`             |
| `accounts`             | select          | Slurm account for job submission               | (dynamic from cluster)     |
| `queues`               | select          | Slurm partition                                | `"ondemand"`               |
| `bc_num_hours`         | number          | Maximum wall time (hours)                      | `1`                        |
| `num_cores`            | number_field    | Number of CPU cores                            | `1`                        |
| `memory`               | number_field    | Memory in GB                                   | `8`                        |
| `node_type`            | select          | Compute node type (CPU, GPU, specific HW)      | Any (shortest wait)        |
| `num_gpus`             | number_field    | Number of GPUs (for GPU node types)             | `0`                        |

## Troubleshooting

<!-- TODO: Add troubleshooting tips -->

## Testing

<!-- TODO: Update with sites where this app has been deployed -->

| Site                          | OOD Version   | Scheduler | Status |
|-------------------------------|---------------|-----------|--------|
| Wake Forest University (DEAC) | <!-- TODO --> | Slurm     | <!-- TODO --> |

## Known Limitations

<!-- TODO: Document any known limitations -->

## Contributing

Contributions are welcome. To contribute:

1. Fork this repository
2. Create a feature branch (`git checkout -b feature/my-improvement`)
3. Submit a pull request with a description of your changes

For bugs or feature requests,
[open an issue](https://github.com/WFU-HPC/OOD-apps.server.gradio-chatbot/issues).

## References

- [Gradio](https://www.gradio.app/) -- the web UI framework used for the
  chat interface
- [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) -- Python
  bindings for llama.cpp, used for LLM inference
- [Meta Llama 3.1](https://ai.meta.com/llama/) -- the LLM family this app
  is configured to use
- [Open OnDemand](https://openondemand.org/) -- the HPC portal framework
- [OOD Batch Connect app development docs](https://osc.github.io/ood-documentation/latest/app-development.html)

### Software Installation

<!-- TODO: Add notes about building the llama-server environment module on your compute nodes -->

## License

[MIT License](LICENSE)

## Acknowledgments

<!-- TODO: Add funding or institutional support information -->

Feel free to use as much or as little of this as you'd like -- we're happy to discuss any of these suggestions or adjust them to better fit your project.

This review is part of the OOD Appverse Affinity Group documentation effort. Thanks for your contributions to the Open OnDemand community!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions