Skip to content

UTF-8 Encoding Latin-1 Bug in HTML files #137

Open
@flooie

Description

@flooie

Unfortunately, we are experiencing occasional crashes on CL from the inability to extract HTML from certain HTML files.

These occur mostly around HTML files downloads from the NY lower courts, with messages like

*Error: 'utf-8' codec can't decode byte 0xe9 in position 437. Having gone thru a number of them I think this is a
latin-1 utf8 encoding bug.

Soo...

I've reviewed the HTML text extraction and found atleast one bug in the unicode/decode encoding part of the loop. If utf8 fails it doesnt bother looping thru the other encodings. But I'm not sure that is the source of the error. So, for now I've updated the error messages, fixed that bug and I expect to wait and see if my improved error messages can catch the bug more directly.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions