Skip to content

Conversation

@shelleydoljack
Copy link
Contributor

@shelleydoljack shelleydoljack commented Dec 18, 2025

Closes #1666

Cleans up some outdated code.
Adds tests for verifying data structure saved to VendorInterface record from form data.
Adds ability to prepend data to the 001 field.
Adds ability to add a subfield to an existing field, given a pattern match in the existing field's specified subfield, or simply add it to existing field regardless.

Vendor Interface view:
Screenshot 2025-12-18 at 2 13 13 PM

Vendor Interface edit view:
Screenshot 2025-12-18 at 2 14 17 PM

I didn't address the fact that the default_data_processor DAG has some hardcoded stuff in it (like remove fields 905, 920, 986 or add field 910 if data is entered in the package name). I also didn't add code to allow users to add a field (not sure if that is a desired feature).

Copy link
Collaborator

@jermnelson jermnelson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good!

@shelleydoljack
Copy link
Contributor Author

Users have noticed an issue with clicking on vendor interfaces that have either an import profile linked to it and/or a file pattern. For example, when clicking on this interface:
Screenshot 2026-01-06 at 2 37 45 PM
The user gets an error page.
The log output shows this error:

[2026-01-06T21:42:02.560+0000] {app.py:1744} ERROR - Exception on /vendor_management/interfaces/152 [GET]
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.11/site-packages/flask/app.py", line 2529, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/flask/app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/airflow/libsys_airflow/plugins/vendor_app/vendor_management.py", line 142, in interface
    return self.render_template(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/flask_appbuilder/baseviews.py", line 342, in render_template
    return render_template(
           ^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/flask/templating.py", line 147, in render_template
    return _render(app, template, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/flask/templating.py", line 130, in _render
    rv = template.render(context)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/jinja2/environment.py", line 1295, in render
    self.environment.handle_exception()
  File "/home/airflow/.local/lib/python3.11/site-packages/jinja2/environment.py", line 942, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/opt/airflow/libsys_airflow/plugins/vendor_app/templates/vendors/interface.html", line 2, in top-level template code
    {% import "vendors/_macros.html" as _macros -%}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/www/templates/airflow/main.html", line 21, in top-level template code
    {% from 'airflow/_messages.html' import show_message %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/flask_appbuilder/templates/appbuilder/baselayout.html", line 2, in top-level template code
    {% import 'appbuilder/baselib.html' as baselib %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/flask_appbuilder/templates/appbuilder/init.html", line 42, in top-level template code
    {% block body %}
  File "/home/airflow/.local/lib/python3.11/site-packages/flask_appbuilder/templates/appbuilder/baselayout.html", line 19, in block 'body'
    {% block content %}
  File "/opt/airflow/libsys_airflow/plugins/vendor_app/templates/vendors/interface.html", line 70, in block 'content'
    <dd class="processing_options">{{ interface.processing_options['prepend_001']['data'] }}</dd>
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/jinja2/environment.py", line 471, in getitem
    return obj[argument]
           ~~~^^^^^^^^^^
jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'prepend_001'
172.28.0.1 - - [06/Jan/2026:21:42:02 +0000] "GET /vendor_management/interfaces/152 HTTP/1.1" 500 1592 "https://sul-libsys-airflow-stage.stanford.edu/vendor_management/vendors/5266" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/26.2 Safari/605.1.15"
172.28.0.1 - - [06/Jan/2026:21:42:02 +0000] "GET /favicon.ico HTTP/1.1" 404 456 "https://sul-libsys-airflow-stage.stanford.edu/vendor_management/interfaces/152" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/26.2 Safari/605.1.15"
127.0.0.1 - - [06/Jan/2026:21:42:07 +0000] "GET /health HTTP/1.1" 200 318 "-" "curl/7.88.1"

@shelleydoljack shelleydoljack marked this pull request as draft January 6, 2026 21:42
@shelleydoljack shelleydoljack force-pushed the t1666-vma-preprocessing branch from 7fa3daf to 4db3001 Compare January 7, 2026 23:58
@shelleydoljack shelleydoljack marked this pull request as ready for review January 8, 2026 00:00
@shelleydoljack
Copy link
Contributor Author

Fixed the issue with rendering interfaces that caused the error above with this commit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

VMA - add preprocessing features

3 participants