Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
306 lines (214 loc) · 8.29 KB

CONTRIBUTING.md

File metadata and controls

306 lines (214 loc) · 8.29 KB

Contributing to vLLM Semantic Router

Thank you for your interest in contributing to the vLLM Semantic Router project! This guide will help you get started with development and contributing to the project.

Table of Contents

Development Setup

Prerequisites

Before you begin, ensure you have the following installed:

  • Docker (or Podman)
  • Make (for build automation)
  • Python 3.10+ (Optional: for training and testing)

Quick Start

  1. Clone the repository:

    git clone https://github.com/vllm-project/semantic-router.git
cd semantic-router

  2. Use the canonical local image workflow:

    make vllm-sr-dev
vllm-sr serve --image-pull-policy never

    For AMD ROCm development:

    make vllm-sr-dev VLLM_SR_PLATFORM=amd
vllm-sr serve --image-pull-policy never --platform amd

    This workflow:

    • Rebuilds the local image
    • Installs the vllm-sr CLI tool
    • Uses the local image only, without pulling a remote fallback

  3. Install Python dependencies (Optional):

    # For training and development
pip install -r requirements.txt

# For end-to-end testing
pip install -r e2e/testing/requirements.txt

Running Tests

Agent Gates

The repository-specific agent harness is indexed in docs/agent/README.md. Treat AGENTS.md as the short entrypoint and docs/agent/* plus tools/agent/* as the durable source of truth. If a real architecture or code/spec gap remains after your change, add or update the durable debt entry indexed from docs/agent/tech-debt/README.md.

Read these first:

Use the agent-specific gates for changed files:

make agent-bootstrap
make agent-validate
make agent-scorecard
make agent-report ENV=cpu CHANGED_FILES="path/one,path/two"
make agent-ci-lint CHANGED_FILES="path/one,path/two"
make agent-ci-gate CHANGED_FILES="path/one,path/two"
make agent-feature-gate ENV=cpu CHANGED_FILES="path/one,path/two"

Use make agent-ci-lint when you want to reproduce the same changed-file lint path that the CI pre-commit workflow runs, including the shared agent bootstrap toolchain and tracked-file codespell check.

ENV=amd is required when platform-specific behavior changed.

Unit Tests

  1. Test Rust bindings:

    make test-binding

  2. Test Go semantic router:

    make test-semantic-router

  3. Test individual classifiers:

    make test-category-classifier
make test-pii-classifier
make test-jailbreak-classifier

Manual Testing

Test different routing scenarios:

# Test model auto-selection
make test-auto-prompt-reasoning
make test-auto-prompt-no-reasoning

# Test PII detection
make test-pii

# Test prompt guard (jailbreak detection)
make test-prompt-guard

# Test tools auto-selection
make test-tools

End-to-End Tests

Ensure both Envoy and the router are running, then:

# Run all e2e tests
python e2e/testing/run_all_tests.py

# Run specific test
python e2e/testing/00-client-request-test.py

# Run tests matching a pattern
python e2e/testing/run_all_tests.py --pattern "0*-*.py"

# Check if services are running
python e2e/testing/run_all_tests.py --check-only

The test suite includes:

  • Basic client request tests
  • Envoy ExtProc interaction tests
  • Router classification tests
  • Semantic cache tests
  • Category-specific tests
  • Metrics validation tests

Development Workflow

Making Changes

  1. Create a feature branch:

    git checkout -b feature/your-feature-name

  2. Make your changes following the project structure and coding standards.

  3. Build and test:

    make agent-report ENV=cpu CHANGED_FILES="path/one,path/two"
make agent-ci-gate CHANGED_FILES="path/one,path/two"
make agent-feature-gate ENV=cpu CHANGED_FILES="path/one,path/two"

  4. Run end-to-end tests:

    make agent-e2e-affected CHANGED_FILES="path/one,path/two"
# Or run a specific profile directly
make e2e-test E2E_PROFILE=ai-gateway

  5. Commit your changes:

    Commit your changes with a clear message, making sure to sign off on your work using the -s flag. This is required by the project's Developer Certificate of Origin (DCO). The repository does not require commit messages to use the PR title classification prefixes.

    git add .
git commit -s -m "clarify PR title guidance"

Debugging

  • View logs: Use vllm-sr logs to view service logs
  • Rust library: Use RUST_LOG=debug environment variable for detailed Rust logs
  • Go library: Use SR_LOG_LEVEL=debug environment variable for detailed Go logs

Code Style and Standards

Code Quality Checks

Before submitting a PR, please run the pre-commit hooks to ensure code quality and consistency. These checks are mandatory and will be automatically run on every commit once installed.

Step 1: Install pre-commit tool

# Using pip (recommended)
pip install pre-commit

# Or using conda
conda install -c conda-forge pre-commit

# Or using homebrew (macOS)
brew install pre-commit

Step 2: Install pre-commit hooks for this repository

# Install pre-commit hooks
pre-commit install

# Run all checks
pre-commit run --all-files
# OR
make precommit-local

Go Code

  • Follow standard Go formatting (gofmt)
  • Use meaningful variable and function names
  • Add comments for exported functions and types
  • Write unit tests for new functionality
  • Keep Go modules tidy: Run make check-go-mod-tidy to verify all modules are tidy
  • Lint Go code: Run make go-lint to check for issues, or make go-lint-fix to auto-fix

Rust Code

  • Follow Rust formatting (cargo fmt)
  • Use cargo clippy for linting
  • Handle errors appropriately with Result types
  • Document public APIs

Python Code

  • Follow PEP 8 style guidelines
  • Use type hints where appropriate
  • Write docstrings for functions and classes

Submitting Changes

  1. Ensure all tests pass:

    make test
python e2e/testing/run_all_tests.py

    The make test command includes:

    • go vet for static analysis
    • check-go-mod-tidy for Go module dependency verification
    • Unit tests for all components

  2. Create a pull request with:

    • A classified PR title using the repository prefixes from .github/PULL_REQUEST_TEMPLATE.md, such as [Doc] Clarify PR title guidance or [Router][CI/Build] Tighten affected test selection
    • Clear description of changes
    • Reference to any related issues
    • Test results and validation steps

  3. Address review feedback promptly

Project Structure

├── candle-binding/          # Rust library for BERT classification
├── src/semantic-router/     # Go implementation of the router
├── src/training/           # Model training scripts
├── e2e/testing/              # End-to-end test suite
├── config/                 # Configuration files
├── docs/                   # Documentation
├── deploy/                 # Deployment configurations
├── Makefile               # Build automation
└── requirements.txt       # Python dependencies

Key Components

  • Candle Binding: Rust library providing BERT-based classification
  • Semantic Router: Go service implementing the Envoy ExtProc interface
  • Training Scripts: Python scripts for fine-tuning classification models
  • Configuration: YAML files defining routing rules and model endpoints

Getting Help

  • Check the documentation
  • Review existing issues and pull requests
  • Ask questions in discussions or create a new issue

License

By contributing to this project, you agree that your contributions will be licensed under the same license as the project (Apache 2.0).