feat: compute eds.ner_crf loss as mean over words

Author: percevalw

changelog.md

@@ -99,6 +99,11 @@
 - The loss of `eds.ner_crf` is now computed as the mean over the words instead of the sum. This change is compatible with multi-gpu training.
 - Having multiple stats keys matching a batching pattern now warns instead of raising an error.
 
+### Changed
+
+- The loss of `eds.ner_crf` is now computed as the mean over the words instead of the sum. This change is compatible with multi-gpu training.
+- Having multiple stats keys matching a batching pattern now warns instead of raising an error.
+
 ### Fixed
 
 - Support packaging with poetry 2.0

edsnlp/pipes/trainable/ner_crf/ner_crf.py

@@ -454,6 +454,9 @@ def collate(self, preps) -> NERBatchInput:
                 k: sum(v) for k, v in preps["stats"].items() if not k.startswith("__")
             },
         }
+        collated["stats"] = {
+            k: sum(v) for k, v in preps["stats"].items() if not k.startswith("__")
+        }
         lengths = [length for sample in preps["lengths"] for length in sample]
         max_len = max(lengths)
         if "targets" in preps: