TypeError: Object of type 'Tensor' is not JSON serializable

[hieu-lam]

Hi,

We have successfully trained the model, but we got a problem while saving the model to a file. The exception is as below:

epoch 54 total loss 0.20950795884999707 0.00021984046049317634
aida-A micro F1: 0.9299655568312285
aida-B micro F1: 0.9415830546265329
msnbc micro F1: 0.9395562356541699
aquaint micro F1: 0.8797202797202797
ace2004 micro F1: 0.8853118712273641
clueweb micro F1: 0.7417355371900826
wikipedia micro F1: 0.7820427483174321
change learning rate to 0.0001
att_mat_diag
tok_score_mat_diag
entity2entity_mat_diag
entity2entity_score_mat_diag
knowledge2entity_mat_diag
knowledge2entity_score_mat_diag
type_emb
cnn.weight
cnn.bias
score_combine.0.weight
score_combine.0.bias
score_combine.3.weight
score_combine.3.bias
save model to model/
Traceback (most recent call last):
File "main.py", line 225, in
ranker.train(conll.train, dev_datasets, config)
File "../DCA/ed_ranker.py", line 1032, in train
self.model.save(self.args.model_path)
File "../DCA/abstract_word_entity.py", line 78, in save
json.dump(config, f)
File "/usr/lib64/python3.6/json/init.py", line 179, in dump
for chunk in iterable:
File "/usr/lib64/python3.6/json/encoder.py", line 430, in _iterencode
yield from _iterencode_dict(o, _current_indent_level)
File "/usr/lib64/python3.6/json/encoder.py", line 404, in _iterencode_dict
yield from chunks
File "/usr/lib64/python3.6/json/encoder.py", line 325, in _iterencode_list
yield from chunks
File "/usr/lib64/python3.6/json/encoder.py", line 437, in _iterencode
o = _default(o)
File "/usr/lib64/python3.6/json/encoder.py", line 180, in default
o.class.name)
TypeError: Object of type 'Tensor' is not JSON serializable

Previously we got the same issue as #7 , but we followed the instruction and got that fixed.

We hope you can assist us with this issue :-)

[YoungXiyuan]

Thank you for your interest in our work and sorry for my late reply.

This issue is mainly caused by the version compatibility problem in the Pytorch. The DCA framework currently does not work with Pytorch 0.4 or higher, you need to install Pytorch 0.3, preferably in a virtual environment so that you can have both versions installed at the same time.

And I also find some possible solutions for this issue if someone have to use Pytorch >= 0.4：
（1）https://discuss.pytorch.org/t/typeerror-tensor-is-not-json-serializable/36065
（2）fossasia/visdom#554

I suspect that tensors stored in variable config of "abstract_word_entity.py#Line75" does not convert to numpy arrays automatically. So you can convert them explicitly as the following manner in "abstract_word_entity.py#Line73-75":

for k, v in self.__dict__.items():
   if not hasattr(v, '__dict__'):
      if isinstance(v, torch.Tensor):
         config[k] = v.cpu().data.numpy()
      else:
         config[k] = v

The above suggestion should hopefully do exactly that automatically, but I haven't been able to take a chance to test it. Glad to hear you found a solution though.

[hieu-lam]

Hm... That doesn't work unfortunately. We've also tried config[k] = v.cpu().numpy() - but the exact same problem still appears.

[YoungXiyuan]

Can you print the data type of the variable v using print(type(v)) before the code line "config[k] = v" ?

I am not sure whether there are some lists of tensors stored in the variable v. While we are able to convert tensors to numpy arrays via "v.cpu().numpy()" or "v.cpu().data.numpy()", converting a list of tensors currently might not be supported.

[hieu-lam]

Hi, I'm sorry for my late reply, here is what we added to the code:

for k, v in config.items():
        print(k, type(v))

And here is the output (none of them is Tensor type, so I guess there must be a list of Tensor somewhere):

......
word_voca <class 'dict'>
entity_voca <class 'dict'>
training <class 'bool'>
_non_persistent_buffers_set <class 'list'>
emb_dims <class 'int'>
freeze_embs <class 'bool'>
hid_dims <class 'int'>
tok_top_n <class 'int'>
margin <class 'float'>
param_copy_switch <class 'bool'>
dr <class 'float'>
gamma <class 'float'>
tok_top_n4ment <class 'int'>
tok_top_n4ent <class 'int'>
tok_top_n4word <class 'int'>
tok_top_n4inlink <class 'int'>
order_learning <class 'bool'>
dca_method <class 'int'>
ent_unk_id <class 'int'>
word_unk_id <class 'int'>
ent_inlinks <class 'dict'>
use_local <class 'bool'>
use_local_only <class 'bool'>
freeze_local <class 'bool'>
saved_log_probs <class 'list'>
rewards <class 'list'>
actions <class 'list'>
order_saved_log_probs <class 'list'>
decision_order <class 'list'>
targets <class 'list'>
record <class 'bool'>
flag <class 'int'>
added_words <class 'list'>
added_ents <class 'list'>
Traceback (most recent call last):
  File "main.py", line 225, in <module>
    ranker.train(conll.train, dev_datasets, config)
  File "../DCA/ed_ranker.py", line 1032, in train
    self.model.save(self.args.model_path)
  File "../DCA/abstract_word_entity.py", line 84, in save
    json.dump(config, f)
  File "/home/trol/.conda/envs/dca/lib/python3.6/json/__init__.py", line 179, in dump
    for chunk in iterable:
  File "/home/trol/.conda/envs/dca/lib/python3.6/json/encoder.py", line 430, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/home/trol/.conda/envs/dca/lib/python3.6/json/encoder.py", line 404, in _iterencode_dict
    yield from chunks
  File "/home/trol/.conda/envs/dca/lib/python3.6/json/encoder.py", line 325, in _iterencode_list
    yield from chunks
  File "/home/trol/.conda/envs/dca/lib/python3.6/json/encoder.py", line 437, in _iterencode
    o = _default(o)
  File "/home/trol/.conda/envs/dca/lib/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
TypeError: Object of type 'Tensor' is not JSON serializable

[hieu-lam]

Hi again,

We've looked into the specific element in those lists and found out that the "targets" is actually the cause - it's a list of Tensor.

for k, v in config.items():
                print(k, type(v))
                if(isinstance(v, list) and (len(v) > 0)):
                    print("list elem type: ",type(v[0]))

Output:

targets <class 'list'>
list elem type:  <class 'torch.Tensor'>

We have some questions regarding to this: is it important to save the "targets" list? Can we ignore it? Can we load the model properly again?

[YoungXiyuan]

Thank you for your provided information and sorry for my late reply.

We have spent some time inspecting the variable "targets" in the mulrel_ranker.py and find out that the variable "target" is associated with a deprecated ablation study experiment and should be useless for the current DCA framework.

I suggest you to reset the variable "targets" to an empty list or not to include it into the dict "config" in the code line #75 of abstract_word_entity.py, and I don't think those operations would affect the model loading during the evaluation phase.

Give it a try (:

[hieu-lam]

Thank you for your suggestion and sorry for my late reply. We've set the "targets" key to an empty list and now everything works just fine :-) We appreciate your assistance.