EXS to ISO/TEI transformation

[git-debase-verbose]

Attached is the Java class that is used in Corpus Services for the task (I changed the file extension to .txt because Github would not allow a .java file in the attachment).

EXS2HIATISOTEI.txt

[berndmoos]

Seems I forgot about this one...

[berndmoos]

Needs to be here in the Partitur-Editor

[berndmoos]

@git-debase-verbose or @sarkipo : This is done now, new preview is upcoming. I don't recall too precisely what we wanted here: I think it was that you were using an unnecessarily inefficient method for turning your EXBs into ISO/TEI. This is now done with a stylesheet which operates on the EXS. It should be very fast. The stylesheet is here:

https://github.com/Exmaralda-Org/exmaralda/blob/master/src/org/exmaralda/tei/xml/exmaraldaEXS2ISOTEI_InelEventBased.xsl

And you can call the method like so:

String yourLanguage = "en"; // replace with something more INEL-like
BasicTranscription exb; // get the exb from somewhere
String filename = "c:\\export\\out.xml";

TEIConverter tc = new TEIConverter();
tc.setLanguage(yourLanguage);
tc.writeINELEventBasedISOTEIToFile(exb, filename);

I checked one result against the ISO/TEI schema, and it validated. Please try and let me know if we need any modifications.

[git-debase-verbose]

Thank you! It works much faster indeed - 3,5 minutes for Nganasan instead of 3,5 hours or so. Two questions:

1. What exactly is the language variable for? The language of the transcription (so, in my test case, it should've been Nganasan)? I see we used "en" in the past anyway...

2. When calling writeINELEventBasedISOTEIToFile, I ran into FileNotFound exceptions:

FileNotFoundException: file:/path/to/corpus/nganasan/song/ChND_061023_SharingSecrets_song/ChND_061023_SharingSecrets_song_tei.xml (Aucun fichier ou dossier de ce nom)

Here is how I set up the filename (doc is the exb processed by DOM, filename is the filename without the path, so ChND_061023_SharingSecrets_song.exb):

String outfile = FilenameUtils.getFullPath(doc.getBaseURI()) + filename.substring(0, filename.lastIndexOf(".")) + "_tei.xml";

It does not matter if I create a new file beforehand or not, the exception is raised anyway.

What kind of path formatting does writeINELEventBasedISOTEIToFile expect? I managed to circumvent it by copying over the contents of the writeINELEventBasedISOTEIToFile method and replacing IOUtilities.writeDocumentToLocalFile with the usual way Corpus Services writes new files. A one-liner featuring writeINELEventBasedISOTEIToFile would be more economic in terms of the code though.

[berndmoos]

What exactly is the language variable for?

For the language of the "text". You can use @xml:lang anywhere optionally, but it is obligatory in the <text> element. So: yes, in your case, the ISO code for Nganasan.

I ran into FileNotFound exceptions:

It should be just the file path (i.e. /path/to/...), not the URL (with protocol file:). If that does not work, it's a Linux thing I must have overlooked. Does it work with just the file path?

[git-debase-verbose]

Yes, it works if I strip the protocol and start treating the path as a string. I'll have to make sure the string would also be a valid path on Windows - but then it's a Corpus Services issue. I presume this one may be closed?

[berndmoos]

If this is about <NSLink> in CoMa, you could try something like:

        File baseFile = new File("/path/to/file.coma"); // absolute file
        String nsLink = "/comm1/transcripte.exb"; // relative path

        Path resolvedPath = baseFile.getParentFile().toPath().resolve(nsLink).normalize();
        File exbFile= resolvedPath.toFile();

This should make the necessary adaptations on every system.

[git-debase-verbose]

It turns out there is a bug affecting texts with multiple speakers - xml:id-s get duplicated:
<tei:annotationBlock xml:id="sc_INEL_1" who="SPK0" start="T0" end="T2">
<tei:annotationBlock xml:id="sc_INEL_1" who="SPK1" start="T3" end="T6">

The ISO/TEI conversion itself works without issues, but when I try to convert ISO/TEIs to JSONs for Tsakorpus, I get the following error:
lxml.etree.XMLSyntaxError: ID sc_INEL_1 already defined

For reference, the way Corpus Services runs ISO/TEI conversion now is via calling writeINELEventBasedISOTEIToFile:
TEIConverter tc = new TEIConverter();
tc.setLanguage(language);
tc.writeINELEventBasedISOTEIToFile(exb, outfile);

I presume it can be fixed rather easily by tweaking the exmaraldaEXS2ISOTEI_InelEventBased stylesheet. Here is an exb with several speakers and an affected ISO/TEI file (both with the added .txt extension, otherwise Github wouldn't let me upload them).

ASS_ChND_190725_Batu_conv.exb.txt

ASS_ChND_190725_Batu_conv_tei.xml.txt

[berndmoos]

Thanks, I'll have a look soon.

[berndmoos]

Should be fixed in the newest preview and webjar version 1.0.12