在执行TaskForSQuADQuestionAnswering训练任务时,经常遇到这样的报错,请问是什么原因导致的?
正在遍历每个问题(样本): 76%|████████████▉ | 16/21 [05:20<01:40, 20.01s/it]
Traceback (most recent call last):
File "TaskForSQuADQuestionAnswering_Train.py", line 210, in
train(config=model_config)
File "TaskForSQuADQuestionAnswering_Train.py", line 81, in train
train_iter, val_iter = data_loader.load_train_data(train_file_path=config.train_file_path)
File "../utils/data_helpers.py", line 766, in load_train_data
postfix=postfix) # 得到处理好的所有样本
File "../utils/data_helpers.py", line 97, in wrapper
data = func(*args, **kwargs)
File "../utils/data_helpers.py", line 649, in data_process
token_to_orig_map = self.get_token_to_orig_map(input_tokens, example[3], self.tokenizer)
File "../utils/data_helpers.py", line 561, in get_token_to_orig_map
token = tokenizer(origin_context_tokens[value_start])
IndexError: list index out of range
在执行TaskForSQuADQuestionAnswering训练任务时,经常遇到这样的报错,请问是什么原因导致的?
正在遍历每个问题(样本): 76%|████████████▉ | 16/21 [05:20<01:40, 20.01s/it]
Traceback (most recent call last):
File "TaskForSQuADQuestionAnswering_Train.py", line 210, in
train(config=model_config)
File "TaskForSQuADQuestionAnswering_Train.py", line 81, in train
train_iter, val_iter = data_loader.load_train_data(train_file_path=config.train_file_path)
File "../utils/data_helpers.py", line 766, in load_train_data
postfix=postfix) # 得到处理好的所有样本
File "../utils/data_helpers.py", line 97, in wrapper
data = func(*args, **kwargs)
File "../utils/data_helpers.py", line 649, in data_process
token_to_orig_map = self.get_token_to_orig_map(input_tokens, example[3], self.tokenizer)
File "../utils/data_helpers.py", line 561, in get_token_to_orig_map
token = tokenizer(origin_context_tokens[value_start])
IndexError: list index out of range