llms.txt script

[Sahu97494]

You can test this out by dropping it into the main directory of the docs-v2 repo and running node scrape_mdx_to_llms.js to see the output in the llms.txt file that is generated. Expected outcome here is to scrape: the title
each section
section name
each link
link title
link url

By submitting a PR to this repository, you agree to the terms within the Auth0 Code of Conduct. Please see the contributing guidelines for how to create and submit a high-quality PR for this repo.

Description

Describe the purpose of this PR along with any background information and the impacts of the proposed change. For the benefit of the community, please do not assume prior context.

Provide details that support your chosen implementation, including: breaking changes, alternatives considered, changes to the API, etc.

If the UI is being changed, please provide screenshots.

References

Include any links supporting this change such as a:

• GitHub Issue/PR number addressed or fixed
• Auth0 Community post
• StackOverflow post
• Support forum thread
• Related pull requests/issues from other repos

If there are no references, simply delete this section.

Testing

Describe how this can be tested by reviewers. Be specific about anything not tested and reasons why. If this library has unit and/or integration testing, tests should be added for new functionality and existing tests should complete without errors.

Please include any manual steps for testing end-to-end or functionality not covered by unit/integration tests.

Also include details of the environment this PR was developed in (language/platform/browser version).

• This change adds test coverage for new/changed/fixed functionality

Checklist

• I have added documentation for new/changed functionality in this PR or in auth0.com/docs
• All active GitHub checks for tests, formatting, and security are passing
• The correct base branch is being used, if not the default branch