Skip to content

Conversation

@leo-amd
Copy link
Contributor

@leo-amd leo-amd commented Nov 4, 2025

Motivation

The primary mlir/utils/jenkins/Jenkinsfile has grown to over 1500 lines, with the vast majority being helper functions defined at the top. This makes the file difficult to read, maintain, and navigate.
This PR refactors all this helper logic into an implicit Shared Library stored in the vars/ directory at the repository root. This greatly simplifies the Jenkinsfile, organizes helper code by purpose, and makes the logic reusable and easier to manage.
This approach also ensures that any PRs modifying the shared library code will use the new code from that same PR for testing, rather than the one from the develop branch.

Technical Details

  • Created vars/ directory: Added a new vars/ directory at the repository root. This is automatically loaded by Jenkins as an implicit library.

  • Grouped Functions into Objects: All Groovy helper functions were moved from the Jenkinsfile and grouped into new "singleton" objects in the vars/ directory:

    • scmUtils.groovy: For gitHealthCheck() and robustScmCheckout().
    • nodeUtils.groovy: For resetGPUs(), checkNodeHealth(), dockerArgs(), get_gpu_architecture(), withHealthyNode(), etc.
    • buildUtils.groovy: For buildProject(), buildCK(), getAndBuildMIGraphX(), etc.
    • testUtils.groovy: For preMergeCheck(), setLitWorkerCount(), parameterSweep(), etc.
    • reportUtils.groovy: For postProcessPerfRes() and archivePerfDB().
    • ciLogic.groovy: For getLabelFromCodepath(), shouldRunFromChip(), resetBuild(), etc.
    • generalUtils.groovy: For splitConfigFile().
  • Updated Jenkinsfile:

    • Removed all helper function definitions.
    • Added an instantiation block at the top to create named objects (e.g., def nodeUtils = nodeUtils(this)).
    • Updated all function calls within the pipeline block to use their new object namespace (e.g., checkNodeHealth() is now nodeUtils.checkNodeHealth()).
  • Handled Global State: The global @Field DOCKER_ARGS_BY_NODE variable remains in the Jenkinsfile. The Jenkinsfile's this (script) context is passed to nodeUtils so it can read/write to this variable using script.DOCKER_ARGS_BY_NODE.

@leo-amd leo-amd requested a review from causten as a code owner November 4, 2025 17:34
@umangyadav umangyadav requested a review from Copilot November 4, 2025 17:40
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the Jenkinsfile by extracting reusable functions into separate utility modules in the vars/ directory. This improves code organization, maintainability, and reusability across Jenkins pipelines.

Key changes:

  • Extracted utility functions from the monolithic Jenkinsfile into five modular files
  • Removed the AgentOfflineException import from the main Jenkinsfile (now in nodeUtils.groovy)
  • Organized functions by domain: build, CI logic, node management, reporting, SCM operations, and testing

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
vars/buildUtils.groovy Build-related functions for compiling projects (buildProject, buildCK, buildMIGraphX)
vars/ciLogic.groovy CI workflow logic including label resolution, codepath filtering, and configuration splitting
vars/nodeUtils.groovy Node health checks, GPU management, Docker configuration, and node selection logic
vars/reportUtils.groovy Performance report post-processing and artifact archiving
vars/scmUtils.groovy Git health checks and robust checkout with retry logic
vars/testUtils.groovy Test execution functions including E2E tests, parameter sweeps, and coverage collection
mlir/utils/jenkins/Jenkinsfile Removed function definitions (moved to utility files) and AgentOfflineException import

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants