How to pick an English title for a Work when using Work/Edition dumps?

[salomvary]

Hi folks,

I'm working on building something using the OpenLibrary data dumps and was wondering what the best strategy would be to pick an English title to represent a Work that was not originally written in English.

Sure, works have a title, with a few problems.

• Works do not expose what language the title has
• I also found no documentation on what language the title is supposed to have
• Some non-English Works have the original, non-English title, like: https://openlibrary.org/works/OL1209318W/Zeit_zu_leben_und_Zeit_zu_sterben
• Some others have an English title, despite being written in another language originally, like: https://openlibrary.org/works/OL985490W/Roman_history
• And, apparently the Work title might be a title that is not the title of any of the Editions of the Work. E.g. the Work titled "Roman history" does not have an Edition titled "Roman history". This makes simple heuristics like "if there is and English-Edition with the same title as the Work, the Work title is likely English" less simple/reliable.

Has anyone successfully solved this problem?

[tfmorris]

If you were my client, I'd counsel you to be really sure you want to lock yourself in to an English-only product, but given that you're convinced of that, why do you care about works at all? Just filter for the English editions and use their titles. If you want to have works, make the work title the most frequent English language edition title.

Works do not expose what language the title has

That's a shortcoming of the data model which was designed with the idea that the work title would always be in English.

I also found no documentation on what language the title is supposed to have

https://openlibrary.org/help/faq/editing#edit-metadata-language

Some non-English Works have the original, non-English title, like:

They are correctly following the guidelines.

Some others have an English title, despite being written in another language originally, like:

They are following the (de facto) previous convention and will probably be updated eventually, but with tens of millions of records, it isn't going to happen very fast.

[salomvary]

If you were my client, I'd counsel you to be really sure you want to lock yourself in to an English-only product, but given that you're convinced of that,

Targeting 1.5 billion English speakers and scaling to 8 billion later is a good first step for something that is just about to launch :)

why do you care about works at all?

I had the impression that often the work title is the best choice if a work has many editions and some/many of them have low quality titles.

Just filter for the English editions and use their titles. If you want to have works, make the work title the most frequent English language edition title.

Sounds like a reasonable strategy, thanks. Might need to do some normalization like count frequency ignoring capitalization but that's an implementation detail.

Works do not expose what language the title has

That's a shortcoming of the data model which was designed with the idea that the work title would always be in English.

Oh got it. I was not aware of the history of changing conventions.

I also found no documentation on what language the title is supposed to have

https://openlibrary.org/help/faq/editing#edit-metadata-language

Some non-English Works have the original, non-English title, like:

They are correctly following the guidelines.

Some others have an English title, despite being written in another language originally, like:

They are following the (de facto) previous convention and will probably be updated eventually, but with tens of millions of records, it isn't going to happen very fast.

Got it. I might as well go in and fix the work title of "Dionis Nicaei, Rerum romanarum à Pompeio Magno, ad Alexandrum Mamaeae filium" ;)

Anyway, thanks for your inputs!

[mekarpeles]

Many editions should have a languages list associated as per the schema:

https://github.com/internetarchive/openlibrary/blob/94cb3f28b71daccf3db7179fca947420e7dd7dd8/openlibrary/schemata/edition.schema.json

[salomvary]

Yup, I'm using that too.

[mekarpeles]

This this your project? https://bookdna.com/

Following with interest.

We also work on https://bookgenomeproject.org/ which was motivated by BookLamp (and the original Book Genome) https://garydrobson.com/2013/04/05/booklamp-and-the-book-genome-project/

See also: https://archive.org/embed/mek-karpeles-nolan-windham-jim-champ

[salomvary]

This this your project? https://bookdna.com/

Yes, and we are launching a new product Soon™️ ;)

Following with interest.

We also work on https://bookgenomeproject.org/ which was motivated by BookLamp (and the original Book Genome) https://garydrobson.com/2013/04/05/booklamp-and-the-book-genome-project/

See also: https://archive.org/embed/mek-karpeles-nolan-windham-jim-champ

Neat, I was not aware of Book Genome. Fun overlap with name. The to-be-launched product has similar goals but takes a different approach, and aims to be a "better Goodreads" :)