fix: don't remap cross-repo GitHub blob URLs to raw - #103
zeitlinger wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adjusts the link-checking remap logic used by lint:links to avoid incorrectly rewriting cross-repo GitHub /blob/ URLs to raw.githubusercontent.com, which can cause false 404s when the /blob/ URL points to a directory (GitHub redirects those to /tree/, but raw does not).
Changes:
- Removes the global remap of all GitHub
/blob/URLs toraw.githubusercontent.com. - Keeps same-repo remaps via
build_remap_args()and retains issue/PR comment anchor stripping.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| echo "--remap" | ||
| # shellcheck disable=SC2016 # single quotes are intentional | ||
| # shellcheck disable=SC2016 # single quotes are intentional: these are regex capture groups, not shell vars | ||
| echo '^https://github.com/([^/]+/[^/]+)/(issues|pull)/([0-9]+)#issuecomment-.*$ https://github.com/$1/$2/$3' |
There was a problem hiding this comment.
build_global_github_args() no longer strips GitHub /blob/ #L… and #:~:text=… fragments for cross-repo URLs. Since lychee runs with include_fragments=true (see .github/config/lychee.toml) and the repo contains external #:~:text links (e.g. tests/test-links.md), this will reintroduce deterministic false failures because Scroll-to-Text fragments are not present in static HTML. Consider keeping fragment-stripping remaps for /blob/ URLs (rewrite to the same https://github.com/<owner>/<repo>/blob/<ref>/<path> without the fragment) while avoiding any remap to raw.githubusercontent.com, which preserves the directory /blob/ → /tree/ redirect behavior that motivated this change.
| echo '^https://github.com/([^/]+/[^/]+)/(issues|pull)/([0-9]+)#issuecomment-.*$ https://github.com/$1/$2/$3' | |
| echo '^https://github.com/([^/]+/[^/]+)/(issues|pull)/([0-9]+)#issuecomment-.*$ https://github.com/$1/$2/$3' | |
| # Cross-repo /blob/ URLs with JS-rendered fragments: | |
| # - Line anchors (#L123, #L10-L20, etc.) | |
| # - Scroll-to-Text fragments (#:~:text=...) | |
| # Lychee can't verify these when include_fragments=true, and the | |
| # fragments don't exist in static HTML. Strip the fragment while | |
| # keeping the URL on github.com (no remap to raw.githubusercontent.com). | |
| echo "--remap" | |
| # Strip line-number anchors on blob URLs | |
| # shellcheck disable=SC2016 # single quotes are intentional: these are regex capture groups, not shell vars | |
| echo '^https://github.com/([^/]+/[^/]+)/blob/([^#]+)#L[0-9]+.*$ https://github.com/$1/blob/$2' | |
| echo "--remap" | |
| # Strip Scroll-to-Text fragments on blob URLs | |
| # shellcheck disable=SC2016 # single quotes are intentional: these are regex capture groups, not shell vars | |
| echo '^https://github.com/([^/]+/[^/]+)/blob/([^#]+)#:~:text=.*$ https://github.com/$1/blob/$2' |
bb2ae48 to
314edf2
Compare
314edf2 to
f7b03b5
Compare
GitHub blob URLs are remapped to raw.githubusercontent.com so lychee can verify #section fragments (GitHub serves blob pages as React apps, so fragments aren't in the HTML — lycheeverse/lychee#1729). This also avoids GitHub's web rate limits (429s). The remap breaks when the URL points to a directory (raw returns 404). Fix by only remapping paths where the last component contains a dot (file extension). Directories like /blob/main/some-dir are left for GitHub to handle normally. Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
f7b03b5 to
ce0d32e
Compare
|
After investigating all 170 cross-repo Rather than adding the extension heuristic here, the simpler fix is to correct those 4 URLs upstream to use Fixed the 3 URLs causing CI failures in java-contrib#2654: c309e075. Keeping this PR open until we confirm that resolves the issue, then closing. |
|
Closing — the fix is to correct the 4 upstream directory URLs to use |
Summary
/blob/URLs toraw.githubusercontent.combuild_remap_args) and issue/PR comment anchor strippingProblem
The global remap breaks when a
/blob/URL points to a directory (e.g.github.com/org/repo/blob/main/some-dir). GitHub handles these via redirect to/tree/, butraw.githubusercontent.comreturns 404 for directories.Found in open-telemetry/opentelemetry-java-contrib#2654 where
https://github.com/open-telemetry/opentelemetry-java-examples/blob/main/javaagent(a directory) was being remapped and failing.Test plan
/blob/directory links