OME 0.1.4

OME 0.1.4

📦 Container Images

The following container images are available:

# OME Manager
docker pull ghcr.io/moirai-internal/ome-manager:v0.1.4

# Model Agent
docker pull ghcr.io/moirai-internal/model-agent:v0.1.4

# OME Agent
docker pull ghcr.io/moirai-internal/ome-agent:v0.1.4

# Multinode Prober
docker pull ghcr.io/moirai-internal/multinode-prober:v0.1.4

⎈ Helm Installation

Option 1: OCI Registry (Recommended)

# Install directly from OCI registry
helm install ome-crd oci://ghcr.io/moirai-internal/charts/ome-crd --version 0.1.4 --namespace ome --create-namespace
helm install ome oci://ghcr.io/moirai-internal/charts/ome-resources --version 0.1.4 --namespace ome

Option 2: GitHub Releases

# Add the OME Helm repository
helm repo add ome https://github.com/sgl-project/ome/releases/download/v0.1.4
helm repo update

# Install OME
helm install ome-crd ome/ome-crd --namespace ome --create-namespace
helm install ome ome/ome-resources --namespace ome

📋 Changelog

📄 Software Bill of Materials

SBOMs are available in both SPDX and CycloneDX formats for:

• Source code
• All container images

🔐 Signatures

All container images are signed with cosign. Verify with:

cosign verify ghcr.io/moirai-internal/ome-manager:v0.1.4   --certificate-identity-regexp=https://github.com/sgl-project/ome/.github/workflows/release.yaml@refs/tags/.*   --certificate-oidc-issuer=https://token.actions.githubusercontent.com

What's Changed

• [misc] cleanup docs by @slin1237 in #197
• fix: benchmark container volume mounts update when base model exists by @carlory in #198
• style: 🔨 optimize import order and more readable. by @yafengio in #191
• [misc] update kimi k2 runtime by @slin1237 in #200
• [misc] update default kimi isvc with ingres disabled by @slin1237 in #201
• [misc] update kimi k2 runtime to remove volumes and env by @slin1237 in #202
• [WIP] Update kimi-k2-pd-rt.yaml by @Atream in #203
• Fix error message for invalid deployment mode by @carlory in #207
• [Bugfix]: verify KEDA ScaledObject CRD is registered before deletion by @bindrad in #206
• [Docs] Fix serving_runtime doc by @bcfre in #218
• Bump cross-env from 7.0.3 to 10.0.0 in /site by @dependabot[bot] in #219
• chart: support for user-defined model agent tolerations by @my-git9 in #221
• [Docs] Improve readability and logical flow of installation guide by @JiangJiaWei1103 in #224
• [Bugfix] Wait for the OME controller Pod to become ready by @JiangJiaWei1103 in #228
• [misc] fix github page version by @slin1237 in #232
• [Misc] Add PVC support to OME agent replica by @beiguo218 in #229
• add acceleratorClass crd by @pallasathena92 in #215
• [misc] upgrade lws version to latest by @slin1237 in #231
• [misc] add oss gpt models by @slin1237 in #233
• [misc] add oss gpt bf16 models by @slin1237 in #234
• [misc] add ut for oss 120b models by @slin1237 in #235
• [BUG]Fix BenchmarkJob Deprecated Storage Command by @YouNeedCryDear in #239
• [Bug]fix openai api violation for accelerator crd by @pallasathena92 in #242
• [BUG]Fix api-backend default value for benchmark job by @YouNeedCryDear in #241
• [Misc] Add checksum upload support in ome agent replica when using OCI as target by @beiguo218 in #240
• Bump actions/download-artifact from 4 to 5 by @dependabot[bot] in #237
• [oep] add proposal for accelerator aware runtime selection by @slin1237 in #129
• Extend ServingRuntime API for Accelerator Support by @pallasathena92 in #246
• [Core] Add local filesystem storage support for model weights by @slin1237 in #253
• feat: support set global hub for chart images by @my-git9 in #251
• add runtime-type helper function test by @pallasathena92 in #249
• [bugfix] add back omitempty on spec and status by @slin1237 in #254
• [bugfix] fix accelerator class Quantity spec by @slin1237 in #256
• [docs] Update Bootstrap utility classes following Bootstrap 5 breaking changes by @yankay in #255
• [bugfix] patch deepseek runtime by @slin1237 in #259
• Support extra volumes and mounts in model-agent by @abatilo in #260
• Extend inferenceService API for Accelerator Support by @pallasathena92 in #258
• [misc] improve hf hub module by @slin1237 in #261
• [core] implement storage interface for multi cloud support by @slin1237 in #262
• [core] implement oci storage by @slin1237 in #263
• [core] implement s3 storage base structure by @slin1237 in #264
• [misc] storage bug fixes and code clean up by @slin1237 in #265
• [core] add s3 and minio storage provider and download option support by @slin1237 in #266
• [core] gcp storage provider base structure by @slin1237 in #267
• Bump actions/setup-go from 5 to 6 by @dependabot[bot] in #272
• Bump actions/setup-node from 4 to 5 by @dependabot[bot] in #271
• Bump actions/github-script from 7 to 8 by @dependabot[bot] in #270
• [Task5]accelerator class controller by @pallasathena92 in #276
• Bump actions/checkout from 4 to 5 by @dependabot[bot] in #245
• [Misc] Add XET-Core Rust Integration for HuggingFace Hub Downloads by @beiguo218 in #277
• [Misc] Add Dockerfile for XET Rust binding linux build by @beiguo218 in #278
• [Misc] Add XET-Core Go binding for HuggingFace Hub Downloads by @pallasathena92 in #279
• [Docs] Add READMEs for Xet binding by @beiguo218 in #280
• [Misc]add static library .a in gitignore by @pallasathena92 in #281
• [core] add pdb controller in ome by @pallasathena92 in #283
• [Misc] Updates in model agent for model deletion: avoid model path deletion when it is referred by other models by @beiguo218 in #284
• [Misc] Migrate to use XET-Core Rust based HF Hub in ome-agent replica by @beiguo218 in #282
• [Core]add runtime selector logic with Awareness AcceleratorClasses by @pallasathena92 in #285
• [Bugfix] Fix list_files fn in XET-Core Rust Hub implementation - implement recursive file listing by @beiguo218 in #286
• Bump cross-env from 10.0.0 to 10.1.0 in /site by @dependabot[bot] in #287
• [Core]Integrate the accelerator‑aware runtime into the InferenceService con… by @pallasathena92 in #289
• [Docs]update code owner by @pallasathena92 in #291
• Bump github/codeql-action from 3 to 4 by @dependabot[bot] in #292
• [OEP]OEP-0005: Model Context Protocol (MCP) Support Design by @YouNeedCryDear in #290
• [Core]Integrate the accelerator‑aware runtime into the InferenceServi… by @pallasathena92 in #293
• [API][Misc] Add new model capabilities and update its determination logic by @beiguo218 in #294
• [Misc] Add more OCI instance types in instance_type_util by @beiguo218 in #295
• [Misc] Add model config parsing support for more model types by @beiguo218 in #296
• [Docs] add readme for AC and update readme for runtime selector by @pallasathena92 in #297
• [Misc][Helm] Add retry logic to scout cache sync by @beiguo218 in #299
• Bump actions/setup-node from 5 to 6 by @dependabot[bot] in #298
• [Helm]add rbac for pdb by @pallasathena92 in #300
• [Bugfix]fix runnerspec during migration by @pallasathena92 in https://github.com/sgl-project/ome/pul...

OME v.0.1.3

🚀 What's New

Kimi K2 Model & Runtime Support

Full support has been added for the Kimi K2 model family, including model parsing and one of the most advanced runtime configurations to date. This release integrates the latest versions of SGLang and SGLang Router, with support for prefill and decode disaggregation, EPLB, and DeepEP.

Model Architecture Expansion

Added support for over 20 new model architectures in the Hugging Face model loader, significantly expanding the range of models that can be deployed and managed.

Multi-Cloud & CoreWeave Support

• Introduced support for storing benchmark results across multiple cloud storage providers.
• Added native support for CoreWeave infrastructure, including H200 instance shape configurations.
• Refactored node shape utilities to support instance types across multiple cloud providers.

Runtime Selection and CRD Enhancements

Serving runtimes now support weighted selection logic based on model format and framework version. These enhancements make the runtime selection process more intelligent and adaptable across diverse model types.

Model Replication & Authentication

Introduced first-class support for replicating models directly from Hugging Face to OCI Object Storage. This includes robust handling of authentication and authorization using both Hugging Face tokens and OCI IAM credentials.

📦 Container Images

The following container images are available:

# OME Manager
docker pull ghcr.io/moirai-internal/ome-manager:v0.1.3

# Model Agent
docker pull ghcr.io/moirai-internal/model-agent:v0.1.3

# OME Agent
docker pull ghcr.io/moirai-internal/ome-agent:v0.1.3

# Multinode Prober
docker pull ghcr.io/moirai-internal/multinode-prober:v0.1.3

⎈ Helm Installation

Option 1: OCI Registry (Recommended)

# Install directly from OCI registry
helm install ome-crd oci://ghcr.io/moirai-internal/charts/ome-crd --version 0.1.3 --namespace ome --create-namespace
helm install ome oci://ghcr.io/moirai-internal/charts/ome-resources --version 0.1.3 --namespace ome

Option 2: GitHub Releases

# Add the OME Helm repository
helm repo add ome https://github.com/sgl-project/ome/releases/download/v0.1.3
helm repo update

# Install OME
helm install ome-crd ome/ome-crd --namespace ome --create-namespace
helm install ome ome/ome-resources --namespace ome

New Contributors

• @JustinTong0323 made their first contribution in #125
• @chengjieyao made their first contribution in #143
• @mupeifeiyi made their first contribution in #150
• @pallasathena92 made their first contribution in #148
• @beiguo218 made their first contribution in #157
• @leprode made their first contribution in #156
• @Iceber made their first contribution in #178
• @carlory made their first contribution in #179
• @yafengio made their first contribution in #190

What's Changed

• Renames SRT to SGLang for clarity by @JustinTong0323 in #125
• [BUG] Fix Inference Service Status Propagation by @YouNeedCryDear in #89
• [bugfix] fix: resolve model agent download loops and timeout issues by @slin1237 in #126
• [core] feat: add multi-cloud storage support for benchmark results by @slin1237 in #127
• [misc] Add support for 20+ new model architectures to HuggingFace mod… by @slin1237 in #128
• feat(helm): Add imagePullSecrets support to ome-resources chart by @abatilo in #131
• [docs] update intallation document by @slin1237 in #130
• Fix YAML syntax error in helm workflow by @abatilo in #133
• Fix typo: rename Scount to Scout by @abatilo in #135
• fix: use non-deprecated node instance type label by @abatilo in #132
• Fix model-agent health check timing to account for startup jitter by @abatilo in #136
• Refactor node shape utilities to support multi-cloud instance types by @abatilo in #134
• Add LWS worker-index 0 to sample DS manifest by @abatilo in #138
• [bugfix] fix nodeshape for unknown vendors by @slin1237 in #141
• [core] adds model node selector to components by @slin1237 in #142
• [bugfix] fix ingress annotation reconciliation by @slin1237 in #144
• [bugfix] Separate OCI Object Storage clients for ome agent by @chengjieyao in #143
• [misc] refactor component cleanup by @slin1237 in #145
• [docs] remove empty pages and fixed styling by @slin1237 in #146
• [docs] fix prism style issue in docs page by @slin1237 in #147
• [Bugfix] make install reports an erros : /bin/sh: 1: read: arg count by @mupeifeiyi in #150
• [Bugfix] Fix models yaml file no such file or directory by @mupeifeiyi in #152
• [DOC]Fix doc architecture diagram SVG title by @YouNeedCryDear in #154
• add modelFormat, modelFramework version compare by @pallasathena92 in #148
• [Misc] replica ome-agent: add support to replicate models from HF to OCI by @beiguo218 in #157
• [Misc] replica ome-agent: minor improvements + cleanup by @beiguo218 in #158
• [Bugfix] try to get baseModelName and protocolVersion from Model and … by @leprode in #156
• [oep] add pvc storage support by @slin1237 in #169
• [core] add pvc storage parsing supporting base model and cluster base… by @slin1237 in #170
• [core] skip model agent task for pvc storage by @slin1237 in #171
• [core] add model metadata parser to ome agent by @slin1237 in #172
• [ci] update ci and makefile by @slin1237 in #173
• [utils] Add CoreWeave H200 shape configuration by @abatilo in #177
• Bump oraclelinux from 9-slim to 10-slim in /dockerfiles by @dependabot[bot] in #174
• fix missing the artifacts dir during make artifacts by @Iceber in #178
• [misc] fix incorrect integration path by @carlory in #179
• Remove training-agent references and update docs by @carlory in #180
• [core] add kimi k2 model support by @slin1237 in #182
• imporve supported runtime sort logic by @pallasathena92 in #183
• [misc] add kimi k2 runtime, isvc, and model by @slin1237 in #184
• [ci] add dev artifact publishing and fix openvex by @slin1237 in #187
• [core] add runtime selector package and deprecate runtime util by @slin1237 in #186
• [chore] update code owner by @slin1237 in #188
• fix: 🔧 Missing pkg/version package. by @yafengio in #190
• fix: Update status using Status().Update method by @carlory in #192
• fix: don't panic when webhook fails to get deploy config by @carlory in #193
• [ci] Fix dev chart handling in workflow by @carlory in #194
• [misc] fix external service being cleaned up and kimi runtime update by @slin1237 in #195
• [chore] update image for 0.1.3 release by @slin1237 in #196

📋 Changelog

Full Changelog: v0.1.2...v0.1.3

📄 Software Bill of Materials

SBOMs are available in both SPDX and CycloneDX formats for:

• Source code
• All container images

🔐 Signatures

All container images are signed with cosign. Verify with:

cosign verify ghcr.io/moirai-internal/ome-manager:v0.1.3   --certificate-identity-regexp=https://github.com/sgl-project/ome/.github/workflows/release.yaml@refs/tags/.*   --certificate-oidc-issuer=https://token.actions.githubusercontent.com

OME v0.1.2

OME v0.1.2

🚀 What's New

• [docs] fix concept site references by @slin1237 in #88
• [bugfix] fix gateway api hyperlink in readme by @slin1237 in #91
• [oep] add multi cloud support proposal by @slin1237 in #93
• Bump actions/configure-pages from 4 to 5 by @dependabot in #61
• Bump softprops/action-gh-release from 1 to 2 by @dependabot in #59
• [misc] add auth interfaces by @slin1237 in #104
• [misc] add oci auth for up, ip, and rp by @slin1237 in #105
• [misc] add aws auth for accesskey, ip, assumerole, wi, etc by @slin1237 in #112
• [misc] ad gcp auth module by @slin1237 in #115
• [misc] ad azure auth module by @slin1237 in #117
• [MISC] Fix Config Runtime by @YouNeedCryDear in #113
• [bugfix] fixes the model agent daemonset deployment volume by @slin1237 in #119
• [docs] updates the serving runtime documentation by @slin1237 in #120
• [bugfix] fix model agent progress logging, deletion, and error handling by @slin1237 in #121
• [bugfix] fix hf token fetch and deletion logic by @slin1237 in #122
• [release] release 0.1.3 by @slin1237 in #123
• Release v0.1.3 by @slin1237 in #124

📦 Container Images

The following container images are available:

# OME Manager
docker pull ghcr.io/moirai-internal/ome-manager:v0.1.2

# Model Agent
docker pull ghcr.io/moirai-internal/model-agent:v0.1.2

# OME Agent
docker pull ghcr.io/moirai-internal/ome-agent:v0.1.2

# Multinode Prober
docker pull ghcr.io/moirai-internal/multinode-prober:v0.1.2

⎈ Helm Installation

Option 1: OCI Registry (Recommended)

# Install directly from OCI registry
helm install ome-crd oci://ghcr.io/moirai-internal/charts/ome-crd --version 0.1.2 --namespace ome --create-namespace
helm install ome oci://ghcr.io/moirai-internal/charts/ome-resources --version 0.1.2 --namespace ome

Option 2: GitHub Releases

# Add the OME Helm repository
helm repo add ome https://github.com/sgl-project/ome/releases/download/v0.1.2
helm repo update

# Install OME
helm install ome-crd ome/ome-crd --namespace ome --create-namespace
helm install ome ome/ome-resources --namespace ome

📋 Changelog

📄 Software Bill of Materials

SBOMs are available in both SPDX and CycloneDX formats for:

• Source code
• All container images

🔐 Signatures

All container images are signed with cosign. Verify with:

cosign verify ghcr.io/moirai-internal/ome-manager:v0.1.2   --certificate-identity-regexp=https://github.com/sgl-project/ome/.github/workflows/release.yaml@refs/tags/.*   --certificate-oidc-issuer=https://token.actions.githubusercontent.com

Full Changelog: v0.1.1...v0.1.2

OME v0.1.1

OME v0.1.1

🚀 What's New

• Bump autoprefixer from 10.4.20 to 10.4.21 in /site by @dependabot in #62
• Fix all doc prefixes by @abatilo in #79
• [misc] fix helm chart and config by @slin1237 in #82
• [MISC] fix ome-manager image by @YouNeedCryDear in #83
• [MISC] Fix Safetensors Version from 1 to 1.0.0 in Runtime by @YouNeedCryDear in #84
• [bugfix] gracefully handle 429 from HF hub by @slin1237 in #85
• [ci] fix release by @slin1237 in #87
• [MISC] Add routerconfig to runtimes and add pd runtimes by @YouNeedCryDear in #78

📦 Container Images

The following container images are available:

# OME Manager
docker pull ghcr.io/moirai-internal/ome-manager:v0.1.1

# Model Agent
docker pull ghcr.io/moirai-internal/model-agent:v0.1.1

# OME Agent
docker pull ghcr.io/moirai-internal/ome-agent:v0.1.1

# Multinode Prober
docker pull ghcr.io/moirai-internal/multinode-prober:v0.1.1

⎈ Helm Installation

Option 1: OCI Registry (Recommended)

# Install directly from OCI registry
helm install ome-crd oci://ghcr.io/moirai-internal/charts/ome-crd --version 0.1.1 --namespace ome --create-namespace
helm install ome oci://ghcr.io/moirai-internal/charts/ome-resources --version 0.1.1 --namespace ome

Option 2: GitHub Releases

# Add the OME Helm repository
helm repo add ome https://github.com/sgl-project/ome/releases/download/v0.1.1
helm repo update

# Install OME
helm install ome-crd ome/ome-crd --namespace ome --create-namespace
helm install ome ome/ome-resources --namespace ome

📋 Changelog

Full Changelog: v0.1.0...v0.1.1

📄 Software Bill of Materials

SBOMs are available in both SPDX and CycloneDX formats for:

• Source code
• All container images

🔐 Signatures

All container images are signed with cosign. Verify with:

cosign verify ghcr.io/moirai-internal/ome-manager:v0.1.1   --certificate-identity-regexp=https://github.com/sgl-project/ome/.github/workflows/release.yaml@refs/tags/.*   --certificate-oidc-issuer=https://token.actions.githubusercontent.com

New Contributors

• @abatilo made their first contribution in #79

OME v0.1.0

OME v0.1.0

🚀 What's New

First release of OME!!!!

📦 Container Images

The following container images are available:

# OME Manager
docker pull ghcr.io/moirai-internal/ome-manager:v0.1.0

# Model Agent
docker pull ghcr.io/moirai-internal/model-agent:v0.1.0

# OME Agent
docker pull ghcr.io/moirai-internal/ome-agent:v0.1.0

# Multinode Prober
docker pull ghcr.io/moirai-internal/multinode-prober:v0.1.0

⎈ Helm Installation

# Add the OME Helm repository
helm repo add ome https://github.com/sgl-project/ome/releases/download/v0.1.0
helm repo update

# Install OME
helm install ome-crd ome/ome-crd --namespace ome --create-namespace
helm install ome ome/ome-resources --namespace ome

📋 Changelog

📄 Software Bill of Materials

SBOMs are available in both SPDX and CycloneDX formats for:

• Source code
• All container images

🔐 Signatures

All container images are signed with cosign. Verify with:

cosign verify ghcr.io/moirai-internal/ome-manager:v0.1.0   --certificate-identity-regexp=https://github.com/sgl-project/ome/.github/workflows/release.yaml@refs/tags/.*   --certificate-oidc-issuer=https://token.actions.githubusercontent.com