Skip to content

SPEC.md

Latest commit

 

History

History
194 lines (152 loc) · 6.91 KB

SPEC.md

File metadata and controls

194 lines (152 loc) · 6.91 KB
Raw

Cascade Deployment Specification v1

Status: Draft v1.0 Last updated: 2026-05-06 Maintained by: Fourier Systems

The Cascade Deployment Specification describes a portable, vendor-neutral YAML format for representing a local AI deployment on a Windows machine. It captures hardware capabilities, recommended model configurations, stack component selection, and performance expectations.

Goals

  1. Portability — Any tool can produce or consume this format
  2. Determinism — The same hardware should produce the same blueprint
  3. Auditability — Every decision is explicit and traceable
  4. Extensibility — Implementations can add fields without breaking consumers

Top-level structure

spec_version: "1.0"

hardware: { ... }       # what the machine has
ollama: { ... }         # primary inference engine config
stack: { ... }          # what software gets deployed
nim_fallback: { ... }   # cloud overflow config (optional)
performance: { ... }    # expected throughput
upgrade_path: "..."     # human-readable upgrade recommendation
scanner: { ... }        # provenance metadata

hardware

hardware:
  gpu:
    name: string                # GPU model name as reported by WMI
    vram_mb: integer            # video memory in megabytes (corrected if WMI under-reports)
    vram_gb: number             # video memory in gigabytes (display-friendly)
    usable_vram_mb: integer     # vram_mb * 0.75 (reserves 25% for KV cache / activations)
    cuda_available: boolean     # true if NVIDIA GPU detected
    driver_version: string      # NVIDIA driver version (e.g. "561.09"); "n/a" for non-NVIDIA
    cuda_version: string        # CUDA version supported by driver (e.g. "12.6"); "n/a" for non-NVIDIA
    driver_ok: boolean          # false if NVIDIA driver is below 452.39 (CUDA 11.3, Ollama minimum)
  cpu:
    name: string                # processor model
    cores: integer              # physical cores
    threads: integer            # logical cores (with hyperthreading)
    speed_ghz: number           # max clock speed
  ram_gb: integer               # total system memory
  storage:
    - drive: string             # drive letter (e.g. "C:")
      free_gb: integer
      total_gb: integer
      used_pct: integer

ollama

ollama:
  primary_model: string         # Ollama model identifier (e.g. "llama3.1:8b-instruct-q4_K_M")
  inference_mode: enum          # "full-gpu" | "hybrid-gpu-cpu" | "cpu-only"
  num_gpu_layers: integer       # number of model layers to offload to GPU
  environment:                  # Windows env vars to set for Ollama
    OLLAMA_NUM_GPU: string
    OLLAMA_NUM_THREAD: string
    OLLAMA_MAX_LOADED_MODELS: string
    OLLAMA_KEEP_ALIVE: string
    OLLAMA_MODELS: string       # must match models_dir
  models_dir: string            # absolute path where Ollama will store model files (maps to OLLAMA_MODELS)
  models_to_pull:               # ordered list of models to download
    - model: string
      gpu_layers: integer
      disk_gb: number

Inference mode semantics

  • full-gpu — All num_gpu_layers fit on the GPU; expected speed is highest
  • hybrid-gpu-cpu — Some layers offload to GPU, the rest run on CPU using system RAM
  • cpu-only — No usable GPU detected; all inference on CPU

Layer count math

num_gpu_layers = min(
    floor(usable_vram_mb / model.mb_per_layer),
    model.total_layers
)

stack

stack:
  tier: enum                    # "lite" | "standard" | "full"
  install_method: enum          # "docker" | "native"
  install_scope: enum           # "system" (admin) | "user" (no admin)
  autostart: boolean            # true = register as Windows service / startup task
  components: [string]          # subset of: ollama, open-webui, flowise, n8n
  port_conflicts:               # pre-flight port availability check
    - service: string           # component name
      port: integer
      in_use: boolean           # true = port occupied; deployment script must abort or remap
  endpoints:
    ollama: string              # default: "http://localhost:11434"
    open_webui: string          # default: "http://localhost:8080"
    flowise: string             # default: "http://localhost:3000"
    n8n: string                 # default: "http://localhost:5678"

Tier rules

Tier RAM threshold Components
lite < 16 GB ollama only
standard 16–31 GB ollama, open-webui
full ≥ 32 GB ollama, open-webui, flowise, n8n

nim_fallback

Optional NVIDIA NIM cloud inference overflow for tasks requiring large models that exceed local capability.

nim_fallback:
  enabled: boolean
  endpoint: string              # e.g. "https://integrate.api.nvidia.com/v1"
  model: string                 # e.g. "meta/llama-3.1-70b-instruct"
  note: string                  # human-readable compliance reminder

For HIPAA/FINRA/GDPR-regulated deployments, set enabled: false.

performance

performance:
  estimated_tokens_per_sec: string   # human range, e.g. "15-25 t/s"
  gpu_layers_assigned: integer
  total_model_layers: integer
  inference_mode: string             # mirrors ollama.inference_mode

Speed estimates by GPU layer count (rough heuristic):

Layers offloaded Estimated speed
All (full-gpu) 25–45 t/s
20+ 15–25 t/s
10–19 8–15 t/s
1–9 4–8 t/s
0 (cpu-only) 2–5 t/s

upgrade_path

Human-readable string. No structured format. Example:

upgrade_path: "RTX 4060 Ti 16GB (~$450 USD) - enables 13B full precision + 34B quantized"

deployment_context

Facts about the environment at scan time that inform HOW to deploy (not what hardware exists).

deployment_context:
  is_admin: boolean             # true if scanner ran with administrator privileges
  install_scope: enum           # "system" | "user" — mirrors stack.install_scope
  docker_available: boolean     # true if Docker daemon responded to `docker info`
  install_method: enum          # "docker" | "native" — mirrors stack.install_method
  autostart: boolean            # whether to register services for system startup

scanner

Provenance metadata.

scanner:
  version: string               # implementation version
  scan_timestamp: string        # ISO 8601 UTC

Reference implementation

The reference implementation is Cascade Scanner (PowerShell). Alternative implementations (Python, Bash for WSL, Go for cross-platform) are welcome.

Versioning

The spec_version field uses semantic versioning. Changes that add optional fields are minor (1.x). Changes that remove fields or change semantics are major (2.x). Consumers should ignore unknown fields and reject configs whose spec_version major component does not match.

License

This specification is published under the MIT License and is freely usable, modifiable, and redistributable.