dev: migrating provenances

[Fabdulla1]

Migrate provenances from the static enum to database-backed records by introducing a provenance model, JSON data source, Mongo repository, and service layer, plus a migration script that loads all records with coordinates and creates lookup indexes. The application wiring now constructs and injects the provenance repository/service, common provenance API endpoints are added for listing, fetching, updating, and children lookups, and fragmentarium archaeology/findspot handling now resolves and validates sites through the provenance service. All dependent areas (corpus, dossiers, fragmentarium, importer, and tests) are updated to use provenance records and schema-based serialization, with new factories and coverage for migration, repository, schema, and service behavior.

[sourcery-ai]

Sorry @Fabdulla1, your pull request is larger than the review limit of 150000 diff characters

[Copilot]

Pull request overview

This PR migrates provenances from a static enum to database-backed provenance records, adding a provenance model/data source, MongoDB repository + service layer, API endpoints, and updating dependent parsing/serialization logic across corpus/fragmentarium/dossiers and tests.

Changes:

• Introduces ProvenanceRecord/GeoCoordinate, JSON-backed seed data, Mongo provenance repository/service, and a migration script with indexes.
• Adds common provenance API endpoints (list/get/update/children) and wires provenance dependencies through app context, repositories, schemas, and parsers.
• Updates fragmentarium/corpus/dossiers codepaths and tests to resolve/validate provenance via ProvenanceService and schema context.

Reviewed changes

Copilot reviewed 79 out of 86 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
ebl/transliteration/domain/atf_parsers/lark_parser/ebl_atf_common.lark	Grammar formatting/normalization.
ebl/transliteration/domain/atf_parsers/lark_parser/ebl_atf_chapter.lark	Grammar formatting/normalization.
ebl/transliteration/domain/atf_parsers/lark_parser/ebl_atf_at_line.lark	Grammar formatting/normalization.
ebl/transliteration/domain/atf_parsers/lark_parser/ebl_atf_abbreviations.lark	Updates provenance abbreviations ordering/entries.
ebl/transliteration/domain/atf_parsers/lark_parser/ebl_atf.lark	Grammar formatting/normalization.
ebl/transliteration/domain/atf_parsers/lark_parser.py	Formatting-only change in parser module.
ebl/tests/transliteration/test_atf_parser.py	Provenance coverage now uses DB-backed records.
ebl/tests/io/test_fragment_importer.py	Schema uses provenance service context for validation/load.
ebl/tests/fragmentarium/test_provenance_route.py	Updates expected /provenances payload to API schema records.
ebl/tests/fragmentarium/test_fragments_search_route.py	Site search tests updated to use provenance records.
ebl/tests/fragmentarium/test_fragment_schema.py	FragmentSchema tests now require provenance service context.
ebl/tests/fragmentarium/test_fragment_archaeology_route.py	Archaeology route tests updated for provenance records.
ebl/tests/fragmentarium/test_colophon_schema.py	Colophon provenance attestation uses provenance records.
ebl/tests/fragmentarium/test_archaeology_schemas.py	Archaeology/Findspot schema tests use provenance service context.
ebl/tests/factories/provenance.py	New provenance factories + default provenance fixtures.
ebl/tests/factories/dossier.py	Dossier factory now chooses from provenance records.
ebl/tests/factories/corpus.py	Corpus factory now chooses from provenance records.
ebl/tests/factories/colophon.py	Colophon provenance values derived from provenance records.
ebl/tests/factories/archaeology.py	Archaeology/findspot factories now use provenance records.
ebl/tests/dossiers/test_dossiers_route.py	Dossiers route tests updated for provenance records.
ebl/tests/dossiers/test_dossiers_repository.py	Dossiers repository tests updated for provenance records.
ebl/tests/dossiers/test_dossier.py	Dossier domain test updated to ProvenanceRecord + schema context.
ebl/tests/corpus/test_parse_chapter.py	Parser tests now pass provenance service through parsing pipeline.
ebl/tests/corpus/test_manuscript.py	Manuscript tests updated to use provenance records.
ebl/tests/corpus/test_corpus.py	Corpus importing-lines test passes provenance service to parsing.
ebl/tests/corpus/test_chapter_schema.py	ChapterSchema load now requires provenance service context.
ebl/tests/corpus/test_chapter_manuscripts_route.py	Manuscript update route tests use provenance record long names.
ebl/tests/corpus/test_chapter_lines_route.py	Chapter import/cache tests updated for new parser signature.
ebl/tests/corpus/test_chapter_display_schema.py	ChapterDisplay schema load now requires provenance service context.
ebl/tests/corpus/test_chapter.py	Chapter domain tests updated for provenance records.
ebl/tests/corpus/test_api_chapter_schema.py	API schemas now load with provenance service context.
ebl/tests/conftest.py	Adds provenance repository/service fixtures; injects service into repos/context.
ebl/tests/common/test_provenance_service.py	New tests for ProvenanceService behavior and caching.
ebl/tests/common/test_provenance_schema.py	New tests for provenance schemas serialization/deserialization.
ebl/tests/common/test_provenance_route.py	New tests for provenance API endpoints.
ebl/tests/common/test_provenance_repository.py	New tests for provenance repository CRUD/children queries.
ebl/tests/common/test_provenance_model.py	New tests for provenance model validation.
ebl/tests/common/test_migrate_provenances.py	New tests for migration script + indexes + coordinates.
ebl/tests/atf_importer/conftest.py	Normalizes env vars for importer test setup.
ebl/io/fragments/importer.py	Fragment import validation optionally uses provenance service context.
ebl/fragmentarium/web/provenances.py	Fragmentarium provenance endpoint switched to provenance service.
ebl/fragmentarium/web/bootstrap.py	Wires provenance service into fragmentarium resources.
ebl/fragmentarium/web/archaeology.py	Archaeology endpoint loads schema with provenance service context.
ebl/fragmentarium/infrastructure/mongo_fragment_repository_get_extended.py	Repo base now requires provenance service.
ebl/fragmentarium/infrastructure/mongo_fragment_repository_get.py	Query/load now uses schema context + passes provenance service to matcher.
ebl/fragmentarium/infrastructure/mongo_fragment_repository_create.py	Create now dumps via schema with provenance service context.
ebl/fragmentarium/infrastructure/mongo_fragment_repository_base.py	Centralizes schema construction with provenance service context.
ebl/fragmentarium/infrastructure/mongo_fragment_repository.py	Updates repository init + schema usage for updates.
ebl/fragmentarium/infrastructure/mongo_findspot_repository.py	Findspot repo now uses schema with provenance service context.
ebl/fragmentarium/infrastructure/fragment_pattern_matcher.py	Site filtering now resolves provenance via provenance service + children lookup.
ebl/fragmentarium/domain/findspot.py	ExcavationSite now aliases ProvenanceRecord.
ebl/fragmentarium/application/archaeology_schemas.py	Schemas resolve site via provenance service context.
ebl/dossiers/infrastructure/mongo_dossiers_repository.py	Dossier schema/repo now resolves provenance via provenance service context.
ebl/dossiers/domain/dossier_record.py	Dossier domain switches provenance type to ProvenanceRecord.
ebl/corpus/web/manuscripts.py	Manual request validation with provenance-aware schema context.
ebl/corpus/web/lines.py	Manual request validation with provenance-aware schema context.
ebl/corpus/web/chapter_schemas.py	Paratext parsing now passes provenance service into transformer.
ebl/corpus/web/bootstrap.py	Corpus wiring now uses CorpusDependencies including provenance service.
ebl/corpus/update_texts.py	Script wiring updated to CorpusDependencies and provenance service.
ebl/corpus/infrastructure/mongo_text_repository_query_fragment.py	Loads attestations with provenance service context.
ebl/corpus/infrastructure/mongo_text_repository_query.py	Loads chapters/manuscripts/dictionary lines with provenance service context.
ebl/corpus/infrastructure/mongo_text_repository_find.py	Loads text/chapter/display schemas with provenance service context.
ebl/corpus/infrastructure/mongo_text_repository_base.py	Repository now stores provenance service dependency.
ebl/corpus/infrastructure/mongo_text_repository.py	Adds constructor to inject provenance service.
ebl/corpus/domain/parser.py	Parsing functions now require provenance service.
ebl/corpus/domain/manuscript.py	Siglum/manuscript provenance now uses ProvenanceRecord.
ebl/corpus/domain/chapter_transformer.py	Transformer resolves provenance via provenance service.
ebl/corpus/application/schemas.py	ManuscriptSchema resolves provenance via provenance service context.
ebl/corpus/application/corpus.py	Introduces CorpusDependencies and threads provenance service into parsing.
ebl/context.py	Context extended to include provenance repository/service.
ebl/common/web/provenances.py	New provenance CRUD/list/children endpoints.
ebl/common/web/bootstrap.py	Registers common /provenances routes + creates indexes.
ebl/common/web/init.py	Package init for common web module.
ebl/common/migrate_provenances.py	Migration script seeds provenances + coordinates + indexes.
ebl/common/infrastructure/mongo_provenance_repository.py	New Mongo repository for provenances + geo bounding query.
ebl/common/infrastructure/init.py	Package init for common infrastructure module.
ebl/common/domain/scopes.py	Adds WRITE_PROVENANCES scope.
ebl/common/domain/provenance_records.json	New canonical provenance record dataset.
ebl/common/domain/provenance_model.py	New provenance domain model + coordinate validation.
ebl/common/domain/provenance_data.py	Builds provenance records from JSON (+ optional geometry overlays).
ebl/common/domain/provenance.py	Removes legacy enum provenance implementation.
ebl/common/application/provenance_service.py	New service layer with caching + update invalidation.
ebl/common/application/provenance_schema.py	New marshmallow schemas for provenance storage/API.
ebl/common/application/provenance_repository.py	New repository interface for provenances.
ebl/app.py	Wires provenance repository/service into app context; registers common routes.

[khoidt]

Please address the following, then tag me again for review (I didn't check all the changes yet).

1. Restructure the new code, use an appropriate subdirectory. The placement should be ebl/provenance/domain/..., ebl/provenance/application/..., etc.
   (not ebl/common/domain/..., ebl/common/application/...).

2. Remove the migration script and related data:

• ebl/common/migrate_provenances.py

• ebl/common/domain/provenance_data.py

• ebl/common/domain/provenance_records.json

• ebl/common/domain/provenance_geometry.json

  and perhaps others that were used solely for the migration, if there are more. If you want to preserve the script and the data for the future (I'm not sure this is needed, but just in case), move it to a dedicated subdirectory in the generic-documentation repo.

3. I found that the /provenances/{id_} endpoint is additionally bound to updating provenance entries in the database, s. my comments. This doesn't seem intended and is a potential hazard that that has to be fixed. Please check the remaining logic. Does everything makes sense to you?

As a general recommendation, I strongly suggest to always go through all changes and revise the logic and functions manually in order to ensure that everything is consequent and there is no redundant code left.

[Copilot]

Pull request overview

Copilot reviewed 73 out of 82 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

ebl/tests/transliteration/test_atf_parser.py:141

• This test no longer exercises provenance parsing for the full set of provenances: it now parametrizes over a small hard-coded tuple from build_provenance_records(), which weakens the purpose of a "coverage" test for the siglum parser’s PROVENANCE terminal. To keep this as a real coverage guard, consider deriving the expected abbreviations from the provenance data source used for the migration (or directly from the grammar terminal pattern) so the test fails if any supported provenance abbreviation becomes unparsable.

[Copilot]

Pull request overview

Copilot reviewed 73 out of 82 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

ebl/tests/transliteration/test_atf_parser.py:141

• This test used to parameterize over the full provenance enum to ensure every provenance abbreviation is accepted by the ATF grammar. Switching to build_provenance_records() reduces that coverage to a small subset of provenances, so grammar regressions for other abbreviations may no longer be detected. Consider sourcing the parametrization from the full provenance data set (e.g., the JSON/migration source or seeded_provenance_service.find_all()) so the test continues to validate complete abbreviation coverage.
  ebl/fragmentarium/web/provenances.py:22
• ebl/fragmentarium/web/bootstrap.py no longer registers a /provenances route, and there are no remaining imports/usages of this ProvenancesResource (it looks superseded by ebl/provenance/web/*). Keeping this file updated but unused adds maintenance overhead and can confuse future readers. Consider removing this module (and any related dead code), or re-wiring it if it is still intended to be used.

[khoidt]

The PR looks good in general, but there are still some issues to address. Please tag me again for a quick review when this is done.

1. Merge conflicts with master has to be resolved before the PR can land

2. Test directory placement is inconsistent with source structure:
   All five new test files (test_provenance_model.py, test_provenance_repository.py, test_provenance_route.py, test_provenance_schema.py, test_provenance_service.py) are placed under ebl/tests/common/ even though the source code is now in ebl/provenance/. Every other module follows the pattern ebl/tests/<module_name>/ (e.g., ebl/tests/corpus/, ebl/tests/fragmentarium/). These should be under ebl/tests/provenance/.

3. Missing test coverage for repository and service methods:
   Files: ebl/tests/common/test_provenance_repository.py, ebl/tests/common/test_provenance_service.py The following methods have no test coverage at all:

   Layer	Untested method
   MongoProvenanceRepository	query_by_long_name
   MongoProvenanceRepository	query_by_abbreviation
   MongoProvenanceRepository	find_by_coordinates
   ProvenanceService	find_by_abbreviation
   ProvenanceService	find_by_id
   ProvenanceService	find_children

4. Multiple repository classes use pass-through __init__ methods that add no behavior
   Files: ebl/corpus/infrastructure/mongo_text_repository.py, ebl/fragmentarium/infrastructure/mongo_fragment_repository.py, ebl/fragmentarium/infrastructure/mongo_fragment_repository_create.py, ebl/fragmentarium/infrastructure/mongo_fragment_repository_get.py, ebl/fragmentarium/infrastructure/mongo_fragment_repository_get_extended.py

   Several classes define:

    def __init__(self, database, provenance_service):
          super().__init__(database, provenance_service)

   without adding validation, conversion, or class-specific setup.

   This is common in cooperative multiple inheritance chains, so it is not inherently wrong. However, here it appears as mixed style because:

   • some repositories use a single meaningful constructor in a base class (MongoTextRepositoryBase, MongoFragmentRepositoryBase),
   • others add extra pass-through constructors at multiple layers,
   • there is no documented rule describing when a pass-through __init__ should be kept.

   Recommendation: keep one constructor where real initialization happens (base class), and remove no-op pass-through constructors unless they are intentionally kept to freeze a public signature. If kept, apply the pattern consistently and type-annotate parameters.

5. Context.get_provenance_service() is redundant and creates mixed access style without any extra logic. Keeping both patterns in parallel (context.get_provenance_service() and context.provenance_service) adds unnecessary indirection and makes the style inconsistent.

   Current mixed usage in this PR:

   • Getter usage: ebl/fragmentarium/web/bootstrap.py:55, ebl/corpus/web/bootstrap.py:40, ebl/corpus/update_texts.py:64
   • Direct attribute usage: ebl/provenance/web/bootstrap.py:13

   Remove get_provenance_service() and standardize all call sites on direct context.provenance_service access.

6. Provenance deserialization logic is duplicated across multiple schema layers
   Files: ebl/corpus/application/schemas.py, ebl/dossiers/infrastructure/mongo_dossiers_repository.py, ebl/fragmentarium/application/archaeology_schemas.py

   The same pattern is implemented in multiple places:

   • read provenance_service from schema context,
   • raise ValidationError("Provenance service not configured.") if missing,
   • resolve by find_by_name,
   • raise ValidationError(f"Invalid provenance: {value}") if missing.

   Examples:

   • ManuscriptSchema.deserialize_provenance() in ebl/corpus/application/schemas.py
   • DossierRecordSchema.deserialize_provenance() in ebl/dossiers/infrastructure/mongo_dossiers_repository.py
   • ProvenanceSiteMixin.deserialize_site() in ebl/fragmentarium/application/archaeology_schemas.py

   This duplication increases drift risk (error messages and behavior can diverge) and makes future provenance-resolution changes harder.

7. Mixed typing style in provenance-related constructors and methods
   Files: ebl/dossiers/infrastructure/mongo_dossiers_repository.py, ebl/corpus/infrastructure/mongo_text_repository.py, ebl/corpus/infrastructure/mongo_text_repository_base.py, ebl/fragmentarium/infrastructure/mongo_findspot_repository.py, ebl/fragmentarium/infrastructure/mongo_fragment_repository*.py, ebl/corpus/web/lines.py, ebl/corpus/web/manuscripts.py, ebl/fragmentarium/web/archaeology.py

   Many newly introduced/updated signatures use untyped provenance_service parameters (and some untyped database parameters), while nearby code uses explicit type hints (Context, ProvenanceService, Database, etc.). This is inconsistent with the project typing standard and reduces readability/tooling value.

8. Redundant unused field in MongoProvenanceRepository
   File: ebl/provenance/infrastructure/mongo_provenance_repository.py

   self._database = database is stored in __init__ but never used elsewhere in the class. It is redundant state and should be removed to keep the repository minimal and clear.

[khoidt]

Running the backend with the frontend application reveals several serious issues. Please tag me again for review after they are fixed. I strongly recommend you to better control the quality of the work without relying on blind vibecoding. Most problems are discoverable via manual checks and comparison with current production UI. I'm attaching a detailed instruction on what should be done in this specific case.

1. 500 Internal Server Error in Library fragments

In Library fragments (any I checked; e.g. /library/K.4840) -> edit -> archeology:

The backend error:

File "falcon/app.py", line 365, in falcon.app.App.__call__
  File "/workspaces/ebl-api/ebl/fragmentarium/web/findspots.py", line 13, in on_get
    resp.media = FindspotSchema().dump(self._repository.find_all(), many=True)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/ebl/fragmentarium/infrastructure/mongo_findspot_repository.py", line 20, in find_all
    return self._schema().load(self._findspots.find_many({}), many=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/schema.py", line 792, in load
    return self._do_load(
           ^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/schema.py", line 999, in _do_load
    raise exc
marshmallow.exceptions.ValidationError: {2487: {'site': ['Invalid provenance: ']}, 2725: {'site': ['Invalid provenance: ']}, 2927: {'site': ['Invalid provenance: ']}, 2936: {'site': ['Invalid provenance: ']}}

---

2. 500 Internal Server Error on Corpus edition intro pages

In corpus edition intro pages -> chapters -> list of manuscripts (e.g. /corpus/L/1/7 -> Standard Babylonian -> lists of manuscripts)

The backend error:

ERROR:root:Unexpected Exception
Traceback (most recent call last):
  File "falcon/app.py", line 365, in falcon.app.App.__call__
  File "/workspaces/ebl-api/ebl/corpus/web/extant_lines.py", line 43, in on_get
    resp.media = ExtantLinesSchema().dump(chapter)["extantLines"]
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/schema.py", line 621, in dump
    result = self._serialize(processed_obj, many=many)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/schema.py", line 589, in _serialize
    value = field_obj.serialize(attr_name, obj, accessor=self.get_attribute)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/fields.py", line 340, in serialize
    value = self.get_value(obj, attr, accessor=accessor)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/fields.py", line 270, in get_value
    return accessor_func(obj, check_key, default)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/schema.py", line 554, in get_attribute
    return get_value(obj, attr, default)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/utils.py", line 283, in get_value
    return _get_value_for_key(obj, key, default)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/utils.py", line 296, in _get_value_for_key
    return getattr(obj, key, default)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/ebl/corpus/domain/chapter.py", line 137, in extant_lines
    return {
           ^
  File "/workspaces/ebl-api/ebl/corpus/domain/chapter.py", line 137, in <dictcomp>
    return {
           ^
  File "<attrs generated methods ebl.corpus.domain.manuscript.Siglum>", line 26, in __hash__
    return hash((
           ^^^^^^
  File "<attrs generated methods ebl.provenance.domain.provenance_model.ProvenanceRecord>", line 30, in __hash__
    return hash((
           ^^^^^^
TypeError: unhashable type: 'list'ERROR:root:Unexpected Exception
Traceback (most recent call last):
  File "falcon/app.py", line 365, in falcon.app.App.__call__
  File "/workspaces/ebl-api/ebl/corpus/web/extant_lines.py", line 43, in on_get
    resp.media = ExtantLinesSchema().dump(chapter)["extantLines"]
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/schema.py", line 621, in dump
    result = self._serialize(processed_obj, many=many)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/schema.py", line 589, in _serialize
    value = field_obj.serialize(attr_name, obj, accessor=self.get_attribute)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/fields.py", line 340, in serialize
    value = self.get_value(obj, attr, accessor=accessor)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/fields.py", line 270, in get_value
    return accessor_func(obj, check_key, default)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/schema.py", line 554, in get_attribute
    return get_value(obj, attr, default)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/utils.py", line 283, in get_value
    return _get_value_for_key(obj, key, default)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/.venv/lib/python3.11/site-packages/marshmallow/utils.py", line 296, in _get_value_for_key
    return getattr(obj, key, default)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ebl-api/ebl/corpus/domain/chapter.py", line 137, in extant_lines
    return {
           ^
  File "/workspaces/ebl-api/ebl/corpus/domain/chapter.py", line 137, in <dictcomp>
    return {
           ^
  File "<attrs generated methods ebl.corpus.domain.manuscript.Siglum>", line 26, in __hash__
    return hash((
           ^^^^^^
  File "<attrs generated methods ebl.provenance.domain.provenance_model.ProvenanceRecord>", line 30, in __hash__
    return hash((
           ^^^^^^
TypeError: unhashable type: 'list'

---

The next two issues might be rather related to the frontend PR or the database migration.
Both possibly require tests the would check the exact content match of the options' menu.

3. Corpus manuscript edit field order doesn't match; indentation and hierarchy is missing.

Compare with production (e.g. in https://www.ebl.lmu.de/corpus/L/1/7/MB/-/edit).

Correct options order and hierarchy

• Standard Text

• Assyria

  • Aššur
  • Dūr-Katlimmu
  • Guzana
  • Ḫarrān
  • Ḫuzirina
  • Imgur-Enlil
  • Kalḫu
  • Kār-Tukulti-Ninurta
  • Khorsabad
  • Nineveh
  • Šibaniba
  • Šubat-Enlil
  • Tarbiṣu

• Babylonia

  • Adab
  • Babylon
  • Bad-Tibira
  • Borsippa
  • Cutha
  • Dilbat
  • Diqdiqqah
  • Dūr-Kurigalzu
  • Eridu
  • Ešnunna
  • Garšana
  • Girsu
  • Girumu
  • Ḫursagkalama
  • Irisagrig
  • Ishān Dhahak
  • Ishān Miziyad
  • Ishān Hāfudh
  • Isin
  • Jamdat Naṣr
  • Kisurra
  • Kiš
  • Kutalla
  • Lagaba
  • Lagaš
  • Larak
  • Larsa
  • Malgium
  • Marad
  • Maškan-šāpir
  • Meturan
  • Meturan (Tell al-Sib)
  • Mukdan
  • Nērebtum
  • Nigin
  • Nippur
  • Pī-Kasî
  • Puzriš-Dagān
  • Sippar
  • Sippar-Amnānum
  • Sulaiymah
  • Šaduppûm
  • Šaḫrīnu
  • Silūqīah
  • Šuruppak
  • Tall Abū Ṣālabīkh
  • Tall Ajrab
  • Tall Imlīḥiyah
  • Tall Jidar
  • Tall Khaibar
  • Tall al-Laḥam
  • Tall Muḥamad
  • Tall al-ʻUbaīd
  • Tall Umm al-ʻAjarib
  • Tall ʻUqair
  • Tulūl al-Baqarāt
  • Tulūl Khaṭāb
  • Tummal
  • Tutub
  • Umma
  • Umm al-Ḥafriyāt
  • Umm al-Wawiya
  • Ur
  • Uruk
  • Yelkhi
  • Zabalam
  • Zaralulu

• Periphery

  • Alalakh
  • Tell el-Amarna
  • ‘Anah
  • Anšan
  • Arrapḫa
  • Ašnakkum
  • Bakr Āwā
  • Bārah
  • Bitwātā
  • Dēr
  • Dugirdkhan
  • Dūr-Untaš
  • Ebla
  • Elam
  • Emar
  • Glay‘a
  • Ḫadatu
  • Hamath
  • Ḫattuša
  • Jericho
  • Kaneš
  • Karkemiš
  • Kimūnah
  • Kumidi
  • Maʾallanate
  • Mardaman
  • Mari
  • Megiddo
  • Mila Mergi
  • Nagar
  • Nereb
  • Nuzi
  • Pašime
  • Persepolis
  • Qatara
  • Qaṭnā
  • Sur Jar‘a
  • Sepphoris
  • Susa
  • Shīshīn
  • Šušarra
  • Tall Bāz Musiyān
  • Tall al-Fakhar
  • Tall Ghaḍaiyrīfah
  • Tall al-Hawa
  • Tall Ibrāhīm Bayis
  • Tell Bāshir
  • Tepe Gotvand
  • Terqa
  • Tigunānum
  • Tikrīt
  • Tripoli
  • Tuttul
  • Ugarit
  • Zawiyah

• Uncertain

---

4. Library search field order doesn't match production; hierarchy is missing.

Compare with production (e.g. in https://www.ebl.lmu.de/library).

Correct options order and data

• Assyria
• Babylonia
• Periphery
• Uncertain
• Adab [Babylonia]
• Alalakh [Periphery]
• ‘Anah [Periphery]
• Anšan [Periphery]
• Arrapḫa [Periphery]
• Ašnakkum [Periphery]
• Aššur [Assyria]
• Babylon [Babylonia]
• Bad-Tibira [Babylonia]
• Bakr Āwā [Periphery]
• Bārah [Periphery]
• Bitwātā [Periphery]
• Borsippa [Babylonia]
• Chughā Mīsh [Periphery]
• Cutha [Babylonia]
• Dēr [Periphery]
• Dilbat [Babylonia]
• Diqdiqqah [Babylonia]
• Dugirdkhan [Periphery]
• Dūr-Katlimmu [Assyria]
• Dūr-Kurigalzu [Babylonia]
• Dūr-Untaš [Periphery]
• Ebla [Periphery]
• Elam [Periphery]
• Emar [Periphery]
• Eridu [Babylonia]
• Ešnunna [Babylonia]
• Garšana [Babylonia]
• Girsu [Babylonia]
• Girumu [Babylonia]
• Glay‘a [Periphery]
• Guzana [Assyria]
• Ḫadatu [Periphery]
• Hamath [Periphery]
• Ḫarrān [Assyria]
• Ḫattuša [Periphery]
• Ḫursagkalama [Babylonia]
• Ḫuzirina [Assyria]
• Imgur-Enlil [Assyria]
• Irisagrig [Babylonia]
• Ishān Dhahak [Babylonia]
• Ishān Hāfudh [Babylonia]
• Ishān Miziyad [Babylonia]
• Isin [Babylonia]
• Jamdat Naṣr [Babylonia]
• Jericho [Periphery]
• Kalḫu [Assyria]
• Kaneš [Periphery]
• Karkemiš [Periphery]
• Kār-Tukulti-Ninurta [Assyria]
• Kayseri [Periphery]
• Khorsabad [Assyria]
• Kimūnah [Periphery]
• Kiš [Babylonia]
• Kisurra [Babylonia]
• Kumidi [Periphery]
• Kutalla [Babylonia]
• Lachish [Periphery]
• Lagaba [Babylonia]
• Lagaš [Babylonia]
• Larak [Babylonia]
• Larsa [Babylonia]
• Maʾallanate [Periphery]
• Malgium [Babylonia]
• Marad [Babylonia]
• Mardaman [Periphery]
• Mari [Periphery]
• Maškan-šāpir [Babylonia]
• Megiddo [Periphery]
• Meturan [Babylonia]
• Mila Mergi [Periphery]
• Mukdan [Babylonia]
• Nagar [Periphery]
• Nereb [Periphery]
• Nērebtum [Babylonia]
• Nigin [Babylonia]
• Nineveh [Assyria]
• Nippur [Babylonia]
• Nuzi [Periphery]
• Pašime [Periphery]
• Persepolis [Periphery]
• Pī-Kasî [Babylonia]
• Puzriš-Dagān [Babylonia]
• Qatara [Periphery]
• Qaṭnā [Periphery]
• Šaduppûm [Babylonia]
• Šaḫrīnu [Babylonia]
• Silūqīah [Babylonia]
• Sepphoris [Periphery]
• Shīshīn [Periphery]
• Šibaniba [Assyria]
• Sippar [Babylonia]
• Sippar-Amnānum [Babylonia]
• Šubat-Enlil [Assyria]
• Sulaiymah [Babylonia]
• Sur Jar‘a [Periphery]
• Šuruppak [Babylonia]
• Susa [Periphery]
• Šušarra [Periphery]
• Tall Abū Ṣālabīkh [Babylonia]
• Tall Ajrab [Babylonia]
• Tall al-Fakhar [Periphery]
• Tall al-Fakhīriyah [Periphery]
• Tall al-Hawa [Periphery]
• Tall al-Laḥam [Babylonia]
• Tall al-ʻUbaīd [Babylonia]
• Tall Bāz Musiyān [Periphery]
• Tall Ghaḍaiyrīfah [Periphery]
• Tall Ibrāhīm Bayis [Periphery]
• Tall Imlīḥiyah [Babylonia]
• Tall Jidar [Babylonia]
• Tall Khaibar [Babylonia]
• Tall Muḥamad [Babylonia]
• Tall ʻUqair [Babylonia]
• Tall Umm al-ʻAjarib [Babylonia]
• Tarbiṣu [Assyria]
• Meturan (Tell al-Sib) [Babylonia]
• Tell Bāshir [Periphery]
• Tell el-Amarna [Periphery]
• Tepe Gotvand [Periphery]
• Terqa [Periphery]
• Tigunānum [Periphery]
• Tikrīt [Periphery]
• Tripoli [Periphery]
• Tulūl Khaṭāb [Babylonia]
• Tummal [Babylonia]
• Tutub [Babylonia]
• Tuttul [Periphery]
• Ugarit [Periphery]
• Umma [Babylonia]
• Umm al-Ḥafriyāt [Babylonia]
• Umm al-Wawiya [Babylonia]
• Ur [Babylonia]
• Uruk [Babylonia]
• Yelkhi [Babylonia]
• Zabalam [Babylonia]
• Zaralulu [Babylonia]
• Zawiyah [Periphery]

[qltysh]

Function with many parameters (count = 6): corpus_dependencies [qlty:function-parameters]

[qltysh]

Function with many parameters (count = 22): context [qlty:function-parameters]

[khoidt]

__init__.py is missing in tests\provenance. You can add it and merge :)