docs(api, docs): port Python API documentation to mkdocs

[ecormany]

Overview

This PR ports the entire Python Protocol API documentation from Sphinx to mkdocs.

Test Plan and Hands on Testing

Sandbox

Changelog

The overall process for building the new site was to run LLM conversion of RST to markdown, inspect the results, and iterate until the new mkdocs site accurately reflected all the content in the edge sandbox.

The commit log for this PR tells a more complete story of the process for conversion.

What this PR touches

• 🆕 Adds files for the new /docs/python-api/ mkdocs project.
  • Markdown source files
  • Copy of image assets
• ✍️ Modifies files in /docs/ that control our top-level docs build.
  • /docs/mkdocs.yml includes the new project.
  • Adds the mkdocstrings plugin for rendering the API reference.
  • Adds a custom griffe plugin for showing API level decorators within the API reference.
  • Adds the macros plugin for substituting the latest API version string in code snippets and prose.
  • Watches the /api/src/opentrons/protocol_api/ directory for changes when running make serve, so if you're editing docstrings you can see the changes automatically.
• 🔗 Updates links in other mkdocs publications that used to go to absolute docs.opentrons.com/v2/... URLs.
• 🔨 Updates Makefile to properly pull in latest docstrings and work around an mkdocs serve bug.
• 🚨 Updates every single public docstring in /api/src/opentrons/ to work with markdown parsing and the new API reference. There should be no code changes except at the whim of black formatting.

What this means for others

• No more Sphinx-style docstrings. The branch name says it all. This is for real. If you are an engineer writing docstrings for a new feature, use Google-style docstrings and markdown formatting. You will never forget the second layer of backticks again. 🎉
• New Python object cross-reference style. The mkdocstrings plugin creates header IDs for each entry in the API reference, which is the fully qualified Python object name. For example, the header ID for transfer() is #opentrons.protocol_api.InstrumentContext.transfer. Because we use the autorefs plugin for mkdocs, you can use a reference-style markdown link to any unique header anywhere in our documentation (including in publications "outside of" the API docs). So the link to transfer() should look like

  [`transfer()`][opentrons.protocol_api.InstrumentContext.transfer]
  

  Since you manually control the link text:
  • Include backticks to get monospace text.
  • Include parentheses for method names and exclude them for attribute names.
  • Write out the class name, e.g. InstrumentContext.transfer() if it feels relevant.
• Python API docs get good search. #finally. No seriously, try it. Type your favorite API feature into the search bar in the sandbox.

To do before deployment

• Figure out how we will handle redirects.
  • The old URLs are bad (all the files with new_ prefixes) and I've wanted to get rid of them forever. Moving from docs.opentrons.com/v2/ to docs.opentrons.com/python-api/ gives us an opportunity to change these filenames (and this PR does).
  • I think redirects are probably best handled in server config rather than in mkdocs. We don't want to output files to a site/v2/ directory at all.
• Inform teams that the change is coming.
• Incorporate Sphinx docs changes made after [email protected] in a follow-up PR
  • (docs)api: using capture_image in 8.8  #20200
  • docs(api): concurrent module actions  #20027
  • RTC-849 dynamic liquid handling

To do after deployment

• Clean out unused RST files in /api/docs/v2/
• Leave behind other /api/docs/ source for archived sites.
• Leave Sphinx tooling in place in case we ever have to rebuild those sites for some reason.

Review requests

Thank you all for wading in to review a massive PR.

• Technical content team: Ensure that the converted docs retain all of the same content of the original docs, without errors.
• Infra and QA: make sure that my changes to uv setup and docs build are cromulent.
• Engineers: Please please please triple-check that I am not messing anything up in code. And let me know if anything about this system seems like a downgrade rather than an upgrade for docs-as-code in the future.

Risk assessment

Medium. Touches every single Python API docstring. Updates docs build process.

[ddcc4]

Hm, I'm assuming mkdocs doesn't support as many of these annotations as sphinx, and that's why you had to change it to English text?

[ecormany]

Yes, that's right. I think nothing is lost in this change. Syntactically, I could never remember the exact Sphinx incantation for these. Semantically, their only target was to become human-readable messages — they were never collated or cross-referenced or anything like that. So here I made the human-readable message directly.

It also has allowed for a little more flexibility. We had some awkward messages of the sort Changed in version 2.x: Removed foo. and now we can just say Removed in version 2.x: Foo.

[ddcc4]

Heh, I don't think I'm going to check every line of this PR, but I assume most of these changes are mechanical?

[ecormany]

Yes. This prompt did great work.

convert this docstring from sphinx-style to google-style. keep all wording exact. replace rst formatting with markdown. replace :ref:, :py:meth:, and :py:obj: crossreferences with markdown autoref style links, for example [transfer()][opentrons.protocol_api.InstrumentContext.transfer]. the link text for methods should have trailing parentheses, like transfer(), and the link text for other objects should not. when hard wrapping a paragraph or continuing an argument definition past one line, always indent the second and subsequent lines by one level (four spaces) more than the first line. replace RST versionadded and versionchanged statements with italic text saying "New in version…" or "Changed in version…" convert rst admonitions to Material for markdown style admonitions, for example beginning with "!!! note" and indenting all admonition content by one additional level.

And mkdocs was good at raising build warnings if links went to nonexistent targets. I cleaned those up by hand.

[ddcc4]

For the other reviewers: Try adding ?w=1 to the Github URL to ignore whitespace changes (so this file would be https://github.com/Opentrons/opentrons/pull/20121/files?w=1#diff-6811e2f4d45b8d159abed031a1b00c50e69ecc5aa7d8e19a674527fb4be3298c ).

But @ecormany: Did you deliberately have to re-wrap these lines for mkdocs? Why is the linebreak so strange-looking here?

[ecormany]

A lot of necessary rewrapping did happen, although this one is admittedly peculiar.

A lot of our old sphinx docstrings had extra indentation, with definitions for things like :param my_super_long_cool_parameter_name: indented by 30+ spaces when exactly 4 would have done.

Mkdocs is very strict on having exactly 4 spaces between Args: and the start of each arg definition, and another 4 spaces for any subsequent lines of the definition. Markdown requires that lists not be further indented, while Sphinx requires that they do.

Ultimately, what matters is that these files 1. output correctly and 2. pass black. Anything else is bonus.

[y3rsh]

Thoughts in the for of a PR, felt like the clearest way #20227