How to enable spacy/transformers pre-trained model for NERs since all predefined recognizers are regex/rules based ?

[himsgpt]

All predefined recognizers are regex/rules based, can we use spacy or transformers-based ML recognizers.
By default predefined recognizers are invoked when calling analyzer_engine, how to enable the pre-trained en_core_web_lg based NERs?

[omri374]

Hi, the predefined setup also uses the en_core_web_lg model and not just rule based recognizers. You can replace this model with other open source ones, add more models as additional recognizers, leverage 3rd party tools like Azure Language Studio, or train your own model with huggingface/stanza/spacy/flair and plug it into Presidio.

References:

1. Using a huggingface model as an additional recognizer: https://github.com/microsoft/presidio/blob/main/docs/samples/python/transformers_recognizer/index.md
2. Using Flair as an additional recognizer: https://github.com/microsoft/presidio/blob/main/docs/samples/python/flair_recognizer.py
3. Using Azure Language Studio (Text Analytics): https://github.com/microsoft/presidio/blob/main/docs/samples/python/text_analytics/index.md
4. Adapting the NLP engine (main NER model): https://microsoft.github.io/presidio/analyzer/customizing_nlp_models/
5. How we shift between approaches in our demo website: https://github.com/microsoft/presidio/blob/main/docs/samples/python/streamlit/presidio_nlp_engine_config.py