Skip to content

Commit 3ee64a9

Browse files
authored
Remove JSON malformation risk in anonymization limitations (#4953)
elastic/docs-content#2695
1 parent 2b751bc commit 3ee64a9

File tree

1 file changed

+0
-1
lines changed

1 file changed

+0
-1
lines changed

docs/en/observability/observability-ai-assistant.asciidoc

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -520,7 +520,6 @@ Anonymization has the following limitations:
520520
* *Performance (NER)*: Running an NER model can add latency depending on the request. To improve performance of the model, consider scaling up your ML nodes by adjusting deployment parameters: increase `number_of_allocations` for better throughput and `threads_per_allocation` for faster individual requests. For details, refer to https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-start-trained-model-deployment[start trained model deployment API].
521521
* *Structured JSON*: The NER model we validated (`elastic/distilbert-base-uncased-finetuned-conll03-english`) is trained on natural English text and often misses entities inside JSON or other structured data. If thorough masking is required, prefer regex rules and craft them to account for JSON syntax.
522522
* *False negatives / positives*: No model or pattern is perfect. Model accuracy may vary depending on model and input.
523-
* *JSON malformation risk*: Both NER inference and regex rules can potentially create malformed JSON when anonymizing JSON data such as function responses. This can occur by replacing text across character boundaries, which may break JSON structure causing the whole request to fail. If this occurs, you may need to adjust your regex pattern or disable the NER rule.
524523

525524

526525
[discrete]

0 commit comments

Comments
 (0)