resourceKilled

[RitaHjz]

Dear Fernando,
Thank you for developing Easse tool. It helps me a lot. However, I’m trying to use SAMSA metric on my output but it fails to compute it. Could you help me to solve it? I tried to download SAMSA but the tool suffers from insufficient info about how to use it and I didn’t understand the code.
Here is the error message:

rita@rita-VirtualBox:~/easse$ easse evaluate -t turkcorpus_test -m 'samsa' -q < easse/resources/data/system_outputs/turkcorpus/test/R
Warning: SAMSA metric is long to compute (120 sentences ~ 4min), disable it if you need fast evaluation.
Loading spaCy model 'en_core_web_md'... Done (76.791s).
Loading from '/home/rita/.local/lib/python3.8/site-packages/easse/resources/tools/ucca-bilstm-1.3.10/models/ucca-bilstm.json'.
[dynet] random seed: 1
[dynet] allocating memory: 512MB
[dynet] memory allocation done.
[dynet] 2.1.2
Loading from '/home/rita/.local/lib/python3.8/site-packages/easse/resources/tools/ucca-bilstm-1.3.10/models/ucca-bilstm.enum'... Done (0.295s).
Loading model from '/home/rita/.local/lib/python3.8/site-packages/easse/resourceKilled

Thanks in advance!

[louismartin]

Hi @RitaHjz,
Thanks for opening this issue. To be fair SAMSA is pretty hard and long to use and we have not implemented the original logic ourselves: https://github.com/eliorsulem/SAMSA

I am not sure what causes your process to be killed, it probably happens outside of the main python thread. It seems to happen when loading a model, maybe it's a memory error?

[RitaHjz]

Hi @louismartin,
Thanks for your quick answer. I tried with few sentences but I have always the same problem :(

[feralvam]

Hi @RitaHjz,

I ran the following command in a fresh installation of EASSE and it worked fine:

easse evaluate -t turkcorpus_test -m 'samsa' -q < easse/resources/data/system_outputs/turkcorpus/test/RM+EX

As Louis mentioned, it seems to be related to lack of memory space when loading the ucca-bilstm model. As such, the number of sentences in the file does not influence there. Have you tried running it in colab, for instance?