What caused this error？ KeyError: 'loss'

[panzhengsen]

I believe there is no problem with my data set construction.
The error message is as follows:

`
['traindata', 'trainxdata']
['validdata', 'validxdata']
BUILDING

controller = GeneralController
modeler = EnasCnnModelBuilder
knowledge = zero
reward = LossAucReward
manager = EnasManager
time budget set to: 24:00:00
env_type = EnasTrainEnv
Trial -1: Start training model with sample_arc...
Traceback (most recent call last):
File "amber_deepsea.py", line 124, in
amb.run()
File "/home/zhouss/AMBER/amber/wrapper.py", line 104, in run
self.env.train()
File "/home/zhouss/AMBER/amber/architect/trainEnv.py", line 418, in train
self.manager.get_rewards(trial=-i, model_arc=None, nsteps=warmup_nsteps)
File "/home/zhouss/AMBER/amber/architect/manager.py", line 544, in get_rewards
verbose=self.verbose,
File "/home/zhouss/AMBER/amber/modeler/child.py", line 743, in fit
prog = ", ".join(["%s=%.4f" % (name, hist.history[name][-1]) for name in to_print])
File "/home/zhouss/AMBER/amber/modeler/child.py", line 743, in
prog = ", ".join(["%s=%.4f" % (name, hist.history[name][-1]) for name in to_print])
KeyError: 'loss'
`
hist.histroy keys： dict_keys([])
I printed hist.histroy keys() to check the keywords and found it was empty

[zj-zhang]

It seems you might have specified a batch size that is larger than the total data size, so that the loss is not computed by call back function by the end of the epoch, because no epoch can run.
Since you already checked hist.history, could you also confirm the value of nsteps in the same for loop? If you see nsteps=0, then changing the batch_size to a smaller value should fix it.

[panzhengsen]

Thank you for your reply.
That is, the batchsize in model_builder is larger than child_batchsize, which has nothing to do with the batchsize in the controller.

[zj-zhang]

@panzhengsen Yes it's the child_batchsize too large. It has nothing to do with the controller's batch size. You can confirm this by examining the values of nsteps.

[panzhengsen]

yes, I printed nsteps and it does show None
I set model_builder to 100 and child_batchsize to 10, but this error is still displayed.
The number of my training set and test set is about a few hundred.
For other parameters, I only modified the shape of the input matrix and the number of output nodes at the end. Could it be caused by improper setting of other parameters?

[zj-zhang]

I see, can you read in your training and validation data (for example, by hdf5 or pickle) without using EncodedGenome? Right now it's not tested for compatibility with Enas modeler (and seems it's incompatible), and thus the issue.

As an example, the input to DeepSEA train/val data are two numpy arrays:
https://github.com/zj-zhang/AMBER/blob/master/examples/amber_deepsea.py#L97

[panzhengsen]

ok.
The function of rna_data_and_feature.py file is to turn fa file into h5py file.
Because the h5py file format of the training set and the test set are the same, I print the output of the test set.
I use read.py to read the test set data in h5py format.
The two python files mentioned above and the read results are placed in result.zip
Because the length of each data in my data set is different, I first fill each sequence with ‘N’ to the same length, replace N with 0.25, and replace the four positions of AUCG with 1 respectively. I use floating-point numbers.
result.zip