Upgrading kreuzberg from 4.2.9 to 4.4.5 (as proposed in #81) changes tesseract's OCR output for image files. In tests/test_kreuzberg.py::test_extract_password[test.png], the same image that reliably extracts as Password123 on 4.2.9 extracts as Password1\n23 on 4.4.5, a line break inserted mid-word.
Confirmed by pulling the #81 branch into an isolated worktree and running the test suite directly (the original CI run's logs had expired, so this was re-verified from scratch rather than trusted at face value). Also confirmed this is not a MANSPIDER-side bug: our own code just calls kreuzberg.extract_file() and does not touch OCR line/word segmentation.
Comparing the two wheels directly (kreuzberg is mostly a compiled Rust extension, so no readable source diff is available) shows the PaddleOCR backend was removed entirely between these versions, a new kreuzberg-cli binary was added, and the package roughly doubled in size (36 MB to 84 MB). This is consistent with an internal OCR pipeline rewrite in that version range, which is the likely cause of the tesseract output change.
This matters beyond the one test: MANSPIDER's core function is regex/content matching over extracted text, including OCR'd images (e.g. screenshots of credentials, scanned documents). If OCR output can silently gain unexpected whitespace or line breaks mid-word, a content filter can silently fail to match text that is genuinely present, with no error or warning, a false negative in a tool whose entire purpose is not missing things.
kreuzberg is now pinned to >=4.2.9,<4.4 in pyproject.toml on dev so this can't drift in accidentally via a routine uv lock --upgrade. This issue tracks investigating further (e.g. whether OcrConfig.tesseract_config can restore the old segmentation behavior, or whether a later kreuzberg release fixes it) before lifting the pin. #81 is being held pending this.
Upgrading
kreuzbergfrom 4.2.9 to 4.4.5 (as proposed in #81) changes tesseract's OCR output for image files. Intests/test_kreuzberg.py::test_extract_password[test.png], the same image that reliably extracts asPassword123on 4.2.9 extracts asPassword1\n23on 4.4.5, a line break inserted mid-word.Confirmed by pulling the #81 branch into an isolated worktree and running the test suite directly (the original CI run's logs had expired, so this was re-verified from scratch rather than trusted at face value). Also confirmed this is not a MANSPIDER-side bug: our own code just calls
kreuzberg.extract_file()and does not touch OCR line/word segmentation.Comparing the two wheels directly (kreuzberg is mostly a compiled Rust extension, so no readable source diff is available) shows the PaddleOCR backend was removed entirely between these versions, a new
kreuzberg-clibinary was added, and the package roughly doubled in size (36 MB to 84 MB). This is consistent with an internal OCR pipeline rewrite in that version range, which is the likely cause of the tesseract output change.This matters beyond the one test: MANSPIDER's core function is regex/content matching over extracted text, including OCR'd images (e.g. screenshots of credentials, scanned documents). If OCR output can silently gain unexpected whitespace or line breaks mid-word, a content filter can silently fail to match text that is genuinely present, with no error or warning, a false negative in a tool whose entire purpose is not missing things.
kreuzbergis now pinned to>=4.2.9,<4.4inpyproject.tomlondevso this can't drift in accidentally via a routineuv lock --upgrade. This issue tracks investigating further (e.g. whetherOcrConfig.tesseract_configcan restore the old segmentation behavior, or whether a later kreuzberg release fixes it) before lifting the pin. #81 is being held pending this.