WIP : Add CI check for broken links

[hardys]

No description provided.

[hardys]

@ranjinimn @fsundermeyer - from local testing I don't think linkspector will work for this as it doesn't consider URLs which contain attributes - can you offer any guidance re alternative ways to catch broken links?

[ranjinimn]

@ranjinimn @fsundermeyer - from local testing I don't think linkspector will work for this as it doesn't consider URLs which contain attributes - can you offer any guidance re alternative ways to catch broken links?

Can you give me an example of the URLs that contain attributes? I generally build the document locally using DAPS and it validates all the broken links.

[hardys]

@ranjinimn @fsundermeyer - from local testing I don't think linkspector will work for this as it doesn't consider URLs which contain attributes - can you offer any guidance re alternative ways to catch broken links?

Can you give me an example of the URLs that contain attributes? I generally build the document locally using DAPS and it validates all the broken links.

@ranjinimn there are many places where we template the URL based on versions, for example:

% linkspector check -s                    
asciidoc/edge-book/welcome.adoc:73:381: 🚫 https://ranchermanager.docs.rancher.com/{rancher-docs-version}/getting-started/installation-and-upgrade/installation-requirements Status:404 Cannot reach link
asciidoc/edge-book/links.adoc:14:27: 🚫 https://documentation.suse.com/sle-micro/{version-operatingsystem}/ Status:301 Cannot reach link
asciidoc/edge-book/links.adoc:15:28: 🚫 https://documentation.suse.com/sle-micro/{version-operatingsystem}/html/Micro-network-configuration/index.html Status:301 Cannot reach link
...

So I think this tool cannot work with attributes - can you expand on your local testing with daps, and how it differs from our current check workflow?

What triggered my investigation here is we currently have some broken links which are not found by building via daps in our CI, for example see https://github.com/suse-edge/suse-edge.github.io/blob/release-3.3/asciidoc/edge-book/releasenotes.adoc which references https://github.com/rancher/rancher/releases/download/v2.11.6/rancher-images.txt - this link no longer works requiring some fixes ref #937

[ranjinimn]

@hardys I checked all the three links in the 3.3 and 3.4 version of the Edge document and they are directing me to the respective content. Let's wait for inputs from @fsundermeyer

[hardys]

@hardys I checked all the three links in the 3.3 and 3.4 version of the Edge document and they are directing me to the respective content. Let's wait for inputs from @fsundermeyer

My point wasn't that those links are broken - only that we can't use this tool to check for actually broken links, because it doesn't understand how to handle links with attributes in them.

I'm looking for alternative approaches which we can use to catch actually broken links in future, suggestions welcome! :)

[fsundermeyer]

TL;DR
The only good solution for the pronblem would be to write something in Ruby that preprocesses the adoc (https://docs.asciidoctor.org/asciidoctorj/latest/extensions/preprocessor/). If that is not an option, either live with the problem or do not use attributes for Links (whatever hurts less).

DAPS contains a link checker that you could use daps -d <DC> linkcheck). It is applied to the XML in which the attributes are resolved. But:

• It currently uses perl-checkbot, which is no longer available for all openSUSE versions.
• Also, it produces a report which you have to check, so it is not suited for script usage
• errors will be reported for the generated XML, not fpr the adoc sources

@hardys What is this linkspector you are using? I am looking for an alternative to perl-checkbot for a long time now...

[hardys]

@fsundermeyer thanks a lot for the insights - the linkspector tool is https://github.com/UmbrellaDocs/linkspector - I found it while searching for github actions which can check links in asciidoc - https://github.com/UmbrellaDocs/action-linkspector

Not using attributes for links may be workable - if we keep all the links in one file we can use sed to replace all-the-versions when bumping, my main concern is people may forget (particularly when doing minor version updates for z-stream releases), perhaps it's possible to write a small script to check though.

I am also aware that our check job is using an old daps image built by @agracey ref https://github.com/suse-edge/suse-edge.github.io/blob/main/.github/workflows/test-deploy.yml#L26 - I don't recall exactly why we needed that but perhaps it's a good time to review and decide if we can align better with how other projects are tested, any guidance on that?