Upgrade to HF `transformers` >= v5

[PeterStaar-IBM]

We need to upgrade docling to use the latest transformers (major) version 5

[geoHeil]

main problem seems to be that we currently still have huggingface-hub>=1.3.0,<2.0 somehow if I understand the problem correctly.

[dolfim-ibm]

The current blocker is that some Docling models fail running with the new transformers. We are trying to address this.

[geoHeil]

would it make sense to stage this into 2 steps

• remove the constraints
• fix the models

I saw some code snippets which are explicitly verifying compliance with a <5 version of transformers i.e. around phi-4.

I think it would be neat if docling itself would be less coupled to a specific version and the user could more flexibly choose based on what models they use to find a compatible setup.

[dolfim-ibm]

We might have to make some workaround, yes. But first let's assess the situation in a few days, it might just all align properly.

[BBC-Esq]

If someone wants to create a checklist of sorts of the various parts of docling that need to be updated and assign tasks I'd be willing to help, just FYI. I'm familiar with the API changes if an outline would help as well?