[Bug] fix: separate CRDs from application manifests

[kinarashah]

Pull Request Description

Problem
Mixing kubectl apply, edit, and server-side apply caused field ownership conflicts during CRD updates. A temporary workaround moved some CRDs to dependencies/, but this was not ideal.

Solution
CRDs are now installed via make install using kubectl create. make deploy now applies only application components. This ensures a cleaner, more consistent installation and upgrade flow.

Related Issues

Resolves: #873

Important: Before submitting, please complete the description above and review the checklist below.

---

Contribution Guidelines (Expand for Details)

We appreciate your contribution to aibrix! To ensure a smooth review process and maintain high code quality, please adhere to the following guidelines:

Pull Request Title Format

Your PR title should start with one of these prefixes to indicate the nature of the change:

• [Bug]: Corrections to existing functionality
• [CI]: Changes to build process or CI pipeline
• [Docs]: Updates or additions to documentation
• [API]: Modifications to aibrix's API or interface
• [CLI]: Changes or additions to the Command Line Interface
• [Misc]: For changes not covered above (use sparingly)

Note: For changes spanning multiple categories, use multiple prefixes in order of importance.

Submission Checklist

• PR title includes appropriate prefix(es)
• Changes are clearly explained in the PR description
• New and existing tests pass successfully
• Code adheres to project style and best practices
• Documentation updated to reflect changes (if applicable)
• Thorough testing completed, no regressions introduced

By submitting this PR, you confirm that you've read these guidelines and your changes align with the project's contribution standards.

[gemini-code-assist]

Hello @kinarashah, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

Hello team, gemini-code-assist here with a summary of this pull request. This PR addresses a bug where mixing different kubectl commands (apply, edit, server-side apply) for managing Kubernetes manifests, particularly CRDs, was causing field ownership conflicts. The core issue stemmed from including CRDs alongside application components in the same deployment process. To fix this, the PR separates the installation of Custom Resource Definitions (CRDs) from the deployment of the main application components. CRDs are now installed first using kubectl create via the make install target, which is more robust for initial CRD setup and avoids the ownership issues seen with apply. The application components, including dependencies like the KubeRay operator itself (but not its CRDs), are then applied separately, typically via make deploy or the subsequent steps in make install. This provides a cleaner and more reliable installation and upgrade path.

Highlights

• CRD/Application Separation: The primary change is the separation of CRD installation from the main application deployment process to prevent kubectl field ownership conflicts.
• Updated Installation Flow: The make install target now explicitly uses kubectl create -k config/crd/ to install CRDs before applying other components. The make uninstall target is also updated to separately delete CRDs.
• Kustomize Structure Changes: The Kustomize configurations have been updated to reflect this separation. Ray CRDs have been moved into the main config/crd path, and the default application kustomization (config/default) no longer includes the CRDs directly.

Changelog

Click here to see the changelog

• Makefile
  • Modified the install target (line 242) to add a step (kubectl create -k config/crd/ || true) specifically for installing CRDs using kubectl create before applying other dependencies.
  • Added a comment (line 241) explaining the use of kubectl create for CRDs.
  • Added a comment (line 244) clarifying that the subsequent kubectl apply step handles KubeRay Operator components from config/dependency.
  • Modified the uninstall target (line 249) to add a step (kubectl delete --ignore-not-found=$(ignore-not-found) -k config/crd) for deleting CRDs separately.
• config/crd/kustomization.yaml
  • Added - ray (line 8) to the resources list to include the Kustomization for Ray CRDs in the main CRD bundle.
• config/crd/ray/kustomization.yaml
  • Added a new file (lines 2-4) defining a Kustomization for the Ray CRDs (ray.io_rayclusters.yaml, ray.io_rayjobs.yaml, ray.io_rayservices.yaml).
• config/default/kustomization.yaml
  • Removed - ../crd (line 22) from the resources list, indicating that the default application Kustomization no longer includes the CRDs.
  • Changed - ../dependency/kuberay-operator (line 28) to - ../kuberay-operator (line 27), updating the path for the KubeRay operator application manifests.
• config/dependency/kustomization.yaml
  • Removed the Ray CRD files (kuberay-operator/crds/ray.io_rayclusters.yaml, kuberay-operator/crds/ray.io_rayjobs.yaml, kuberay-io_rayservices.yaml) (lines 5-7) from the resources list, as these CRDs are now managed separately.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request effectively separates CRD installation from application manifest deployment, addressing issues with field ownership conflicts. The changes in the Makefile and Kustomize configurations are clear and move towards a cleaner deployment process.

One key area for discussion is the method chosen for CRD installation (kubectl create ... || true), particularly concerning CRD updates and error handling. Please see the detailed comment on the Makefile.

Summary of Findings

• CRD Installation and Update Strategy: The use of kubectl create -k config/crd/ || true for installing CRDs does not handle updates to existing CRD definitions and may mask genuine creation errors. This could impact the CRD upgrade process and hide potential issues during installation.

Merge Readiness

The pull request makes good progress in separating CRD management. However, there's a high-severity concern regarding the CRD installation strategy (kubectl create ... || true) in the Makefile. Specifically, its implications for handling CRD updates and the potential for masking installation errors need to be addressed or clarified.

I recommend discussing the points raised in the review comment before merging. As a reviewer, I am not authorized to approve pull requests; please ensure further review and approval from authorized maintainers after addressing the feedback.

[varungup90]

why do we need to change kuberay-operator dir? That can still be part of dependency along with envoy-gateway.

[varungup90]

Reason to not split make install into two commands was that in public release, we want to keep installation simple for end users and not ask them to do make install. Public release has one command for dependency.

# Install component dependencies
kubectl apply -f "https://github.com/vllm-project/aibrix/releases/download/v0.3.0/aibrix-dependency-v0.3.0.yaml" --server-side

# Install aibrix components
kubectl apply -f "https://github.com/vllm-project/aibrix/releases/download/v0.3.0/aibrix-core-v0.3.0.yaml"

[varungup90]

Using create will have problem in future if we want to update the crd definition for ray objects. Thats why we changed to use --server-side in v0.3.0 release which does throw error for large size object.