Since the new pipeline outputs a unique .jsonl file containing both textual content and metadata, working exclusively with textual data may be less comfortable:
- Forced download of metadata that will be discarded
- Forced usage of a
jsonl-compatible parser in order to extract content
For now, we should provide a tool that splits up a given OSCAR Schema v2 file into a OSCAR Schema v1.2-compatible file.
Since the new pipeline outputs a unique
.jsonlfile containing both textual content and metadata, working exclusively with textual data may be less comfortable:jsonl-compatible parser in order to extract contentFor now, we should provide a tool that splits up a given OSCAR Schema v2 file into a OSCAR Schema v1.2-compatible file.