Skip to content

Conversation

@petecheslock
Copy link
Collaborator

@petecheslock petecheslock commented Oct 15, 2025

Problem

The previous system queried the GitHub API during builds to fetch release information, which added complexity, required tokens to avoid rate limits, and slowed down builds. The release workflow was unclear and builds had external dependencies that could fail.

Solution

Replace API-based syncing with a simple static YAML file that gets updated manually (via a script) when cutting a release. The build process now just reads from this committed file—no API calls, no tokens, no external dependencies during builds.

Changes

Release Management

  • Added components-data.yaml as the single source of truth for:
    • Release version, date, and URLs
    • Component list with descriptions, versions, and metadata
    • Repository locations and branch/tag information
  • Created sync-release.mjs - a one-time script that queries the GitHub API to update the YAML file when cutting a release
  • Removed release-info.json and the old github-api-utils.js (no longer needed)
  • Removed GITHUB_TOKEN from CI workflow since builds no longer make API calls
  • Updated component-configs.js to load from YAML instead of hardcoded arrays
  • Added js-yaml dependency for YAML parsing

Guide Versioning

  • Modified guide-generator.js to fetch guides from release tags (e.g., v0.3.0) instead of main branch
    • Ensures guides match the actual released version users are installing
    • Users get accurate installation instructions for their version
  • Disabled path versioning logic since we're not using Docusaurus versioning yet (but prepared for future use)

Source Attribution

  • Updated footer callouts to differentiate between versioned and non-versioned content:
    • Versioned docs (components, guides) link to the specific release tag
    • Show clear guidance on where to find latest development changes
    • Non-versioned docs (community) link to main branch
  • Fixed YAML escaping and newline handling issues in footer rendering

Documentation

  • Comprehensive README overhaul with new sections:
    • "How It Works" - explains the remote content system architecture
    • "File Structure" - visual guide to the directory layout
    • "Cutting a New Release" - step-by-step workflow with examples
    • "Adding New Components" - YAML-based instructions
    • "Adding New Content Sources" - template-based approach
    • "Troubleshooting" - common issues and solutions table
  • Updated CONTRIBUTING.md to reference README and use YAML examples
  • Cleaned up docs/community/events.md - removed past events (Oct 2025 IBM TechXchange, All Things Open)

Release Workflow

When a new llm-d release is published, the process is now crystal clear:

# Step 1: Update the YAML file (queries GitHub API once)
cd remote-content/remote-sources
node sync-release.mjs
git diff components-data.yaml      # Review changes

# Step 2: Commit and deploy (triggers build that reads YAML)
git add components-data.yaml
git commit -m "Update to llm-d vX.Y.Z"
git push

Key distinction: The script queries the API → commits a static file → the build reads that file. No API calls during builds.

Content Synchronization Strategy

The documentation now uses three distinct synchronization strategies based on content type:

1. Components (/docs/architecture/Components/)

Sync from individual component release tags

  • Each component's README syncs from its specific version tag mentioned in the llm-d release
  • Examples:
    • llm-d-inference-schedulerv0.3.2
    • llm-d-routing-sidecarv0.3.0
    • llm-d-modelservicellm-d-modelservice-v0.2.10 (special format)
  • Why: Provides version-accurate documentation for each component as it existed at release time

2. Guides (/docs/guide/)

Sync from llm-d/llm-d release tag (e.g., v0.3.0)

  • All installation and setup guides sync from the main llm-d release version
  • Why: Ensures guides match the exact version users are installing

3. Community (/docs/community/)

Always sync from main branch

  • Community documents (CODE_OF_CONDUCT, CONTRIBUTING, SECURITY, SIGS) always reflect current upstream
  • Why: Provides up-to-date contribution guidelines and policies regardless of release version

… information fetching in components generator. Update deploy workflow to include GITHUB_TOKEN for authenticated requests.

Signed-off-by: Pete Cheslock <[email protected]>
- Introduced `components-data.yaml` to serve as a single source of truth for component configurations and release information.
- Updated `component-configs.js` to load component data from the new YAML file, eliminating the need for dynamic GitHub API calls.
- Refactored the components generator to utilize static data from the YAML file, simplifying the documentation generation process.
- Removed the `github-api-utils.js` file as its functionality is no longer needed.

This change enhances maintainability and reduces complexity in managing component data.

Signed-off-by: Pete Cheslock <[email protected]>
- Updated the README.md to include detailed instructions for updating to a new release, including an automated sync script.
- Modified components-data.yaml to reflect the latest release version and date, ensuring accurate documentation.
- Improved the guide generator to fetch content from the specific release version instead of the main branch, aligning documentation with released versions.
- Enhanced source callouts in documentation to differentiate between release versions and main branch content.

Signed-off-by: Pete Cheslock <[email protected]>
- Removed the GITHUB_TOKEN environment variable from the deployment workflow to streamline the build process.
- Revised the README.md to clarify steps for cutting a new release, including detailed instructions for updating the YAML file and committing changes.
- Improved clarity on the manual operations required for syncing and deploying documentation updates.

Signed-off-by: Pete Cheslock <[email protected]>
…lify documentation path generation. The function was previously intended for future Docusaurus versioning but is currently unused, allowing for a more straightforward approach to path handling.

Signed-off-by: Pete Cheslock <[email protected]>
@petecheslock petecheslock self-assigned this Oct 15, 2025
@netlify
Copy link

netlify bot commented Oct 15, 2025

Deploy Preview for elaborate-kangaroo-25e1ee ready!

Name Link
🔨 Latest commit 03470ea
🔍 Latest deploy log https://app.netlify.com/projects/elaborate-kangaroo-25e1ee/deploys/68f14a6c82c642000857f201
😎 Deploy Preview https://deploy-preview-109--elaborate-kangaroo-25e1ee.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

- Updated README.md to clarify the synchronization process for components, guides, and community docs, including handling of special tag formats.
- Refactored component configuration management in component-configs.js to streamline loading of component and release information from YAML.
- Improved components-data.yaml to include version tags for components, ensuring accurate versioning in documentation.
- Enhanced sync-release.mjs to transform version tags for specific repositories and update components with the latest release information.

Signed-off-by: Pete Cheslock <[email protected]>
- Updated the createStandardTransform function to use 'ref' (version or branch) instead of just 'branch' when transforming content. This change enhances the flexibility of repository configurations by allowing for more accurate versioning in the transformation process.

Signed-off-by: Pete Cheslock <[email protected]>
@petecheslock petecheslock force-pushed the feat/refactor-release-sync branch from 74db722 to 93787d9 Compare October 15, 2025 20:07
- Revised CONTRIBUTING.md to reflect changes in component configuration file from `component-configs.js` to `components-data.yaml`.
- Enhanced README.md to clarify the remote content system, including detailed instructions for syncing and managing components, guides, and community documentation.
- Removed the outdated remote-content/README.md file to streamline documentation.
- Updated repo-transforms.js with a TODO comment regarding future link fixes after the next llm-d release.

Signed-off-by: Pete Cheslock <[email protected]>
- Updated the createSourceCallout function to accept an optional mainReleaseVersion parameter, allowing for more informative documentation callouts based on version tags.
- Added scenarios to differentiate between documentation for version tags with and without the main release version, improving clarity for users.
- Modified the generateComponentRemoteSource function to pass the mainReleaseVersion from components data, ensuring accurate versioning in generated content.

Signed-off-by: Pete Cheslock <[email protected]>
…se versions

- Changed the output filename for transformed documentation from `00_architecture.mdx` to `architecture.mdx`.
- Implemented synchronization with the specific release version defined in `components-data.yaml`, ensuring that documentation aligns with the released version rather than the main branch.
- Updated URL generation to utilize the release version for fetching content, enhancing the accuracy of the documentation transformation process.

Signed-off-by: Pete Cheslock <[email protected]>
@petecheslock petecheslock force-pushed the feat/refactor-release-sync branch from 4c593e7 to f11409c Compare October 16, 2025 19:34
@jjasghar jjasghar merged commit 3857cac into llm-d:main Oct 16, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants