Context word not improving score

[SingTeng]

Hi,

I did not find a date of birth recognizer in the supported entities (https://microsoft.github.io/presidio/supported_entities/) so I create a custom one.
I tried to use the context word to increase detection accuracy, but the context word does not seem to work for me.

==============
My code:

text = "My date of birth is 2000/01/01. I am thinking of going to the cinema on 06/05/2023 to watch the movie"

my dob recognizer, simply recognize date

dob_pattern = Pattern(name="dob_pattern (weak)",regex=r"(\d{2}(-|/)\d{2}(-|/)\d{4})|(\d{4}(-|/)\d{2}(-|/)\d{2})", score = 0.01)

I add context word, hoping it call differentiate between normal date and dob

dob_recognizer = PatternRecognizer(supported_entity="DOB", patterns = [dob_pattern], context = ["date of birth","dob", "d.o.b."])

calling the analyzer

result = dob_recognizer.analyze(text=text, entities=["DOB"])
print("Result:")
print(result)

looking at decision process

decision_process = result[0].analysis_explanation

pp = pprint.PrettyPrinter()
print("Decision process output:\n")
pp.pprint(decision_process.dict)

==========
My result:

Result:
[type: DOB, start: 20, end: 30, score: 0.01, type: DOB, start: 72, end: 82, score: 0.01]
Decision process output:

{'original_score': 0.01,
'pattern': '(\d{2}(-|\/)\d{2}(-|\/)\d{4})|(\d{4}(-|\/)\d{2}(-|\/)\d{2})',
'pattern_name': 'dob_pattern (weak)',
'recognizer': 'PatternRecognizer',
'score': 0.01,
'score_context_improvement': 0,
'supportive_context_word': '',
'textual_explanation': None,
'validation_result': None}

==============

It seems to have recognize both date as DOB, having the same score.
My expected result would be, the first date should have higher score as DOB than second date.
I might have done something wrong?
Please advise.
Thanks.

[omri374]

Hi @SingTeng, the current context aware enhancement logic uses single words or tokens, and not phrases (also see #1043). Therefore, passing a context term such as "date of birth" would result in a mismatch. Passing ["date", "birth"] instead should work. Could you please try and let us know if it helps?

[SingTeng]

After passing single token, it still didn't work.
Then I realize the key issue is, it doesn't work when I apply the recognizer to 'analyze' directly.
I need to add the recognizer and use the AnalyzerEngine.
Then it works.

Updated code below:

from presidio_analyzer import PatternRecognizer
from presidio_analyzer import AnalyzerEngine, RecognizerRegistry

text = "I am thinking of going to the cinema on 06/05/2023 to watch the movie. My date of birth is 2000/01/01."

regex=r"(\d{2}(-|\/)\d{2}(-|\/)\d{4})|(\d{4}(-|\/)\d{2}(-|\/)\d{2})"
dob_pattern = Pattern(name="dob_pattern (weak)",regex=regex, score = 0.01)

dob_recognizer = PatternRecognizer(supported_entity="DOB", patterns = [dob_pattern], context = ["birth"],)

# this does not work
# result = dob_recognizer.analyze(text=text, entities=["DOB"])
# print("Result:")
# print(result)

registry = RecognizerRegistry()
registry.add_recognizer(dob_recognizer)
analyzer = AnalyzerEngine(registry=registry)

result = analyzer. Analyze(text=text, entities=["DOB"], language="en")
print("Result:")
print(result)

[omri374]

Yes, that's correct. The context mechanism is part of the AnalyzerEngine and not the recognizer, as it can support inter-recognizer context (e.g. if one recognizer identified a person's name, it increases the likelihood of an adjacent text being someone's title)

[ta378]

Hello,
I have created a custom Date of Birth recognizer. I tried to use the context word to increase detection accuracy, but the context word does not seem to work for me. Every date is extracted as DOB.
My code:

dob_regex=r"\b(?!.*(?:birth|(?:date)|birthday|dob|born)\W+(?:\w+\W+))((0?[1-9]|1[0-2])[-\/](0?[1-9]|[12][0-9]|3[01])[-\/](19|20)?\d{2}|(0?[1-9]|1[0-2])(0?[1-9]|[12][0-9]|3[01])(19|20)\d{2}|(?:January|February|March|April|May|June|July|August|September|October|November|December)\s+(?:0?[1-9]|[12][0-9]|3[01])(?:st|nd|rd|th)?,\s+(19|20)?\d{2}|(19|20)\d{6})\b"
dob_pattern =Pattern(name="dob_pattern (weak)",regex=dob_regex,score=0.01)
dob_recognizer = PatternRecognizer(supported_entity="DOB", patterns = [dob_pattern],context=["birth"],)
dob_text = "His date of birth is 07/17/1988. On 09/27/2022 he was gone missing."
registry = RecognizerRegistry()
registry.add_recognizer(dob_recognizer)
analyzer = AnalyzerEngine(registry=registry)
results = analyzer.analyze(text=dob_text,entities=["DOB"], language="en")
print("Results:")
[(dob_text[result.start:result.end], result.start, result.end,result.score) for result in results]

My Result:
Results:
[('07/17/1988', 21, 31, 0.4), ('09/27/2022', 36, 46, 0.4)]

[omri374]

The word "birth" falls into the window of the second sentence as well. You could change the window size this way by changing parameters on the context_aware_enhancer:

from presidio_analyzer.context_aware_enhancers import LemmaContextAwareEnhancer
enhancer = LemmaContextAwareEnhancer(context_prefix_count=1, context_suffix_count=1)
analyzer = AnalyzerEngine(registry=registry,context_aware_enhancer=enhancer)
results = analyzer.analyze(text=dob_text,entities=["DOB"], language="en", return_decision_process=True)
[result.to_dict() for result in results]

This could be a bug though, as windows larger than 1 still puts "birth" into the window. We will look into this.

[ta378]

I still get same results:
`[{'entity_type': 'DOB', 'start': 21, 'end': 31, 'score': 0.4, 'analysis_explanation': {'recognizer': 'PatternRecognizer', 'pattern_name': 'dob_pattern (weak)', 'pattern': '\\b(?!.*(?:birth|(?:date)|birthday|dob|born)\\W+(?:\\w+\\W+))((0?[1-9]|1[0-2])[-\\/](0?[1-9]|[12][0-9]|3[01])[-\\/](19|20)?\\d{2}|(0?[1-9]|1[0-2])(0?[1-9]|[12][0-9]|3[01])(19|20)\\d{2}|(?:January|February|March|April|May|June|July|August|September|October|November|December)\\s+(?:0?[1-9]|[12][0-9]|3[01])(?:st|nd|rd|th)?,\\s+(19|20)?\\d{2}|(19|20)\\d{6})\\b', 'original_score': 0.01, 'score': 0.4, 'textual_explanation': None, 'score_context_improvement': 0.39, 'supportive_context_word': 'birth', 'validation_result': None}, 'recognition_metadata': {'recognizer_name': 'PatternRecognizer', 'recognizer_identifier': 'PatternRecognizer_140207845331856'}}, {'entity_type': 'DOB', 'start': 36, 'end': 46, 'score': 0.01, 'analysis_explanation': {'recognizer': 'PatternRecognizer', 'pattern_name': 'dob_pattern (weak)', 'pattern': '\\b(?!.*(?:birth|(?:date)|birthday|dob|born)\\W+(?:\\w+\\W+))((0?[1-9]|1[0-2])[-\\/](0?[1-9]|[12][0-9]|3[01])[-\\/](19|20)?\\d{2}|(0?[1-9]|1[0-2])(0?[1-9]|[12][0-9]|3[01])(19|20)\\d{2}|(?:January|February|March|April|May|June|July|August|September|October|November|December)\\s+(?:0?[1-9]|[12][0-9]|3[01])(?:st|nd|rd|th)?,\\s+(19|20)?\\d{2}|(19|20)\\d{6})\\b', 'original_score': 0.01, 'score': 0.01, 'textual_explanation': None, 'score_context_improvement': 0, 'supportive_context_word': '', 'validation_result': None}, 'recognition_metadata': {'recognizer_name': 'PatternRecognizer', 'recognizer_identifier': 'PatternRecognizer_140207845331856'}}]