Script to test notebooks

[hrobarts]

Draft script to test if notebooks run correctly with latest version of CIL

• Finds all notebooks in specified folders
• Makes a tempory copy of the notebook
• If data is loaded, updates the data filepath to /mnt/share/ (this could be amended if we want to get data from somewhere else)
• In exercises, replaces # %load lines with the correct snippet and removes cells with ...
• Tests notebooks with pytest --nbmake (chose not to use nbval because most of our notebooks are not saved rendered)

Things to think about

• Should this live in CIL or CIL-Demos repo?
  • How to trigger? Takes ~1hr to run all demos so don't run every time
  • If on CIL we could add a script to render notebooks for docs. If on CIL demos we could save the rendered docs somewhere accessible to CIL.
  • We are also using notebooks from CIL-User-Showcase in docs that should be tested. Would be simpler to do this all on CIL.
  • If we render and save the outputs of notebooks automatically can we avoid saving rendered versions on github?
• How to get data?
  • I've updated where the how-to files save data - now they download files to cache so will download first time on GHA machine but then be accessible whenever GHA are run on the same machine.
  • Could do similar in demos folder - some files are on zenodo and accessible from dataexample.download_data() but most are not. Would be a lot of work to make all of them accessible. Can we mount the shared drive on the GHA machine?
• Which environment?
  • Tested so far with cil_demos env from Check if environment has skimage before running tests CIL#2102

[casperdcl]

fyi:

• data cache env var: Zeiss Reader test with Walnut file CIL#2098 (comment)
• render notebooks in docs: https://github.com/TomographicImaging/CIL/blob/master/docs/mkdemos.py

[lauramurgatroyd]

If it lives in CIL it could be triggered by a release (or if we had one some kind of prerelease or release candidate tag)
However we wouldn't want it to make a red cross that makes it look like cil is failing!

[casperdcl]

Nope, to be "triggered by a [CIL] release", "it" doesn't need to "live in CIL". One option is a workflow_{call,dispatch}, another is a clone à la TomographicImaging/CIL#2102 (comment). Probably both.

[lauramurgatroyd]

Closes #110