Consolidate tags

[MHindermann]

Consolidate tags and tag allocation to bechmarks:

• Add tag metadata (description, data schema)
• Allocate tags to individual benchmarks
• Refactor datalcass type where required depending on tag(s)

[sorinmarti]

Currently used tags are divided in 5 groups but saved as a flat structure

Description of Source Type (What context do you send per request?):

• book-pages
• index-cards
• letter-pages
• manuscript-pages
• article-pages
• essay-pages

Structure of source (How is the context structured?):

• text-like
• list-like

Text Type (If it is about text [also if from images]: How is the text produced?)

• printed-source
• typed-source
• handwritten-source

Century/epoch (from which century does the context stem [originally]?):

• century-15th
• century-18th
• century-19th
• century-20th

Languages (which languages are text in?):

• language-german
• language-english

Entry Type (what structure do you extract from the context?)

• company-entries
• bibliographic-entries
• ner-entries

Task (What is done by the LLM?):

• ner-extraction,
• metadata-extraction,
• transcription,
• classification

Misc

• test_benchmark

The above listed tags are the ones currently used. Changeable in meta.json of each benchmark.

The create_benchmark.py CLI proposes to use at least one of every group if applicable.

Do you want a separate list to choose from @MHindermann ? Actually used tags are listed in collected_results/tags.csv and collected_results/tags.json

[MHindermann]

I've collected all tags from README.MD and collected_results/tags.json:

3. Tags

• Structured tags from specific categories:
  • Source Type: index-cards, letter-pages, manuscript-pages, book-pages, article-pages, essay-pages, registers, lists
  • Structure: text-like, list-like, table-like, mixed
  • Text Type: handwritten-source, typed-source, printed-source
  • Century: century-15th, century-16th, century-17th, century-18th, century-19th, century-20th
  • Languages: language-german, language-english
  • Entry Type: company-entries, bibliographic-entries, ner-entries
  • Task: ner-extraction, metadata-extraction, transcription, classification
  • Misc: test-benchmark

Then there are lots of language tags missing (for Library Cards). Should we use ISO 639-2 for languages instead?