Skip to content

dgdr.mdx

Latest commit

 

History

History
120 lines (92 loc) · 5.49 KB

dgdr.mdx

File metadata and controls

120 lines (92 loc) · 5.49 KB
Raw
title DGDR Templates
subtitle Ready-to-apply DynamoGraphDeploymentRequest manifests for profiling and generating DynamoGraphDeployments.

A DynamoGraphDeploymentRequest (DGDR) describes a model, workload, hardware, and optional latency targets. Dynamo profiles that intent and generates the DynamoGraphDeployment (DGD) that serves traffic. Use these templates when you want Dynamo to choose a deployment configuration instead of writing the complete DGD yourself.

Each manifest is embedded from examples/deployments/dgdr/. Open an example, use the copy button, and adjust its model, backend, hardware, and namespace before applying it.

Prerequisites

  • Install the Dynamo Kubernetes platform and operator.
  • Create hf-token-secret in the target namespace when the model requires authentication.
  • For namespace-restricted operator installations, set hardware.gpuSku, hardware.vramMb, and hardware.numGpusPerNode explicitly.

Release installations select the matching profiler image when spec.image is omitted. For a local operator build without a known release version, set spec.image to a compatible dynamo-planner image.

Apply a template with:

export NAMESPACE=dynamo-cloud
kubectl apply -f rapid.yaml -n ${NAMESPACE}

Generate a DGD

Uses simulated performance estimates to generate a DGD without benchmarking candidate engines on real GPUs. 
<Code src="../../../../../examples/deployments/dgdr/rapid.yaml" title="rapid.yaml" language="yaml" maxLines={0} />
Deploys and benchmarks candidate configurations before selecting and generating the DGD. 
<Code src="../../../../../examples/deployments/dgdr/thorough.yaml" title="thorough.yaml" language="yaml" maxLines={0} />
Adds a Planner configuration so the generated disaggregated deployment can adjust prefill and decode replicas at runtime. 
<Code src="../../../../../examples/deployments/dgdr/planner.yaml" title="planner.yaml" language="yaml" maxLines={0} />
Uses SGLang and a 32-GPU budget for a large Mixture-of-Experts model. Adapt its hardware, scheduling, model-cache, and SLA settings before applying it. 
<Code src="../../../../../examples/deployments/dgdr/moe-sglang.yaml" title="moe-sglang.yaml" language="yaml" maxLines={0} />

Review and Customize the Generated DGD

Sets `autoApply: false` so the request stops after generation instead of creating the DGD. 
<Code src="../../../../../examples/deployments/dgdr/review-before-deploy.yaml" title="review-before-deploy.yaml" language="yaml" maxLines={0} />
Merges an environment-variable override into the generated frontend component. Overrides can modify generated components but cannot add a new topology component. 
<Code src="../../../../../examples/deployments/dgdr/generated-dgd-override.yaml" title="generated-dgd-override.yaml" language="yaml" maxLines={0} />
Mounts a pre-populated model cache into the profiling job and generated workers. Set the backend runtime image and create the PVC before applying the request. 
<Code src="../../../../../examples/deployments/dgdr/model-cache.yaml" title="model-cache.yaml" language="yaml" maxLines={0} />

Test and Inspect Profiling

Replaces model-serving workers with mock workers for testing routing, profiling, and Planner behavior without serving the model on GPUs. 
<Code src="../../../../../examples/deployments/dgdr/mocker.yaml" title="mocker.yaml" language="yaml" maxLines={0} />
Replaces the profiling output volume with the existing `dynamo-pvc` claim so plots, logs, configurations, and raw profiling data remain available after the Job completes. 
<Code src="../../../../../examples/deployments/dgdr/profiling-artifacts.yaml" title="profiling-artifacts.yaml" language="yaml" maxLines={0} />

Inspect the Result

Watch the request until it reaches Deployed or Ready:

kubectl get dgdr <request-name> -n ${NAMESPACE} -w

For a request with autoApply: false, extract the generated DGD:

kubectl get dgdr <request-name> -n ${NAMESPACE} \
  -o jsonpath='{.status.profilingResults.selectedConfig}' > generated-dgd.yaml

See Auto Deploy with DGDR for the task-oriented workflow and the DGDR Reference for field definitions, lifecycle phases, and validation behavior.