[Feature]: Integrations with other backends via hOcr (naive implementation of easyOcr backend inside)

[coffepowered]

Describe the proposed feature

Hi, I see there are a few issues on the board proposing integrations of new backends.

I wondered how difficult this would be to do naively: it turns out that's doable, here's the result of a quick-and-dirty plugin I created in a couple of hours. I converted a nonreadable sample pdf using OCRmyPDF with easyOCR backend:

I basically created a hocr output from easyOCR result's object.However I am not sure if this is a suitable approach or has fundamental limitations that prevent this kind of integration from succeeding.

I expect any OCR to provide bounding boxes+text (at least) that can be then expressed in hOcr format.

Is there some profound or semantic limitations I am unaware of that make the reconstruction of hOcr format difficult?

[coffepowered]

Oh, nevwrmind about the easyocr stuff, I see there's an official plugin (I'll look into that).

I still would be grateful of @jbarlow83 can comment on the issue in general: what are the difficulties/nuances of supporting multiple backends- besides the thin wrappinfg layer I was so clumsy to try reimplementing myself?

[jbarlow83]

I wrote easyocr as direct to PDF, and then significantly improved hocr to PDF so that the various OCR engines that return some flavor of JSON will have an easy path to convert to hocr. So hocr will become the one common backend for any OCR engine. hocr is also reasonably editable to support the use case of people who want to change the final OCR (e.g. manual spelling correction etc.) and the API supports this now.

Currently right to left text (Arabic-Hebrew-Farsi) is a problem, and I'm not sure there's a glyphless font solution that will work for these languages on all PDF viewers (they have serious issues in Tesseract PDF too). I had pretty limited time for that effort and had to stop at that point.

But if RTL isn't a concern, then you just need to do a fairly simple JSON to hocr conversion at this stage.

[coffepowered]

Thanks for the response, it's appreciated to have some help and orientation with this.

I wrote easyocr as direct to PDF, and then significantly improved hocr to PDF so that the various OCR engines that return some flavor of JSON will have an easy path to convert to hocr. So hocr will become the one common backend for any OCR engine

So, as far as I understand, if you were to rewrite to easyOCR plugin today I'd rather use hocr instead, is this accurate?
I am asking because I am thinking about a flexible way to support other open OCR model/engines: one super simple way would be to "convert" to results to be easyOCR-like then use your official plugin (which is way better engineered than my POC code).

Currently right to left text (Arabic-Hebrew-Farsi) is a problem, and I'm not sure there's a glyphless font solution that will work for these languages on all PDF viewers (they have serious issues in Tesseract PDF too)

Great insight, fortunately RTL languages are definitely out of my use cases.

[jbarlow83]

Yes, you definitely could convert some other OCR engine's JSON to EasyOCR JSON, and use the existing plugin.

[mkyt]

FYI, I wrote a simple plugin, ocrmypdf-AppleOCR, to support OCR with text recognition engine in Apple Vision Framework on macOS by converting the OCR output to hocr format.
I think it's relatively easy to adopt other engines using the same approach.

[laundmo]

As this seems like the correct place to ask: Is it possible to write a plugin which does not need to implement OcrEngine.generate_pdf?
It seems redundant to be able to generate hocr and pdf, when OCRmyPDF already has the ability built in to generate a PDF from hocr.
I would appreciate some input on this, as both the code and the docs are not clear under which circumstances OcrEngine.generate_hocr and OcrEngine.generate_pdf are required - do i always need both? Will OCRmyPDF generate the PDF from hocr if OcrEngine.generate_pdf is not provided?

[jbarlow83]

OcrEngine.generate_hocr and generate_pdf are not both required - you have to implement at least one of them, and then you can use pdf-renderer hocr or sandwich to select them respectively. I readily concede the plugin interface is not that clear and needs work, here and in other places.

Since hocr output is rare and this is a bit of an issue for people who want to try better OCR engines, I'm working on a simple internal format, so we would instead go hocr -> internal format -> pdf.