IN 1438 - v1 of tag exporting notebook

[ghukill]

Purpose and background context

This PR turns the stubbed tag_export.py notebook into a functional notebook. This is 90% a port from the POC notebook in timdex-notebooks and 10% improvement on that. This first pass is establishing a baseline similar to the POC to continue testing in a deployed context.

There are currently no meaningful tests. I'm feeling a little uncertain what kind of tests would be valuable for such a notebook, and would like to return to that in a future PR. Having this notebook functional will unblock us to test the notebook in a deployed fashion: configurations, permissions, memory/CPU resources tuning, etc.

How can a reviewer manually see the effects of these changes?

With those caveats above, it is functional as-is!

1- Set Dev1 AWS TimdexManagers credentials in terminal and set env vars:

TDA_LOG_LEVEL=DEBUG
WARNING_ONLY_LOGGERS=asyncio,botocore,urllib3,s3transfer,boto3,MARKDOWN
TIMDEX_DATASET_LOCATION=s3://timdex-extract-dev-222053980223/dataset_scratch/prod-clone

2- Start notebook with Makefile, that opens it in edit mode:

make edit-notebook-tag-export

3- View as "app" mode which is consistent with how users will see it. Click this button in the lower-right:

4- Experiment with MARC tags (e.g. try 650,655 or 918,985,900), and limits, etc! Just be aware that omitting the limit, or setting it large, can take quite awhile. I think the full Alma "current records" is about 3.9m, and takes about 10 minutes to download and parse tags.

As I type this... realizing we may have an opportunity for caching records we've downloaded; maybe For a future ticket 😎.

5- Perform an export. Try a small set like:

Then try to export it using the "Download" UI in the lower-right:

Includes new or updated dependencies?

YES

Changes expectations for external applications?

NO

What are the relevant tickets?

• https://mitlibraries.atlassian.net/browse/IN-1438

[ehanson8]

Looks good and works as expected! The unit test question for marimo notebooks will be interesting to discuss, I agree that it'll be easier to talk about after we get a better sense of the process. I'm also interested in exploring functional code outside of the notebook as we previously discussed, that could be unit tested more traditionally. But getting this out is the excellent and necessary first step!

[ghukill]

Looks good and works as expected! The unit test question for marimo notebooks will be interesting to discuss, I agree that it'll be easier to talk about after we get a better sense of the process. I'm also interested in exploring functional code outside of the notebook as we previously discussed, that could be unit tested more traditionally. But getting this out is the excellent and necessary first step!

100% agree @ehanson8.

In this scenario, 99% of the work we do is just TDA functionality and I wouldn't want to test that. BUT, I'd love to test the MARC parsing using the marcalyx library, and that would be nice external code (outside the notebook) that we could test.