Nan values and -inf during training with Akita for HiC prediction

[ananyapavuluri]

Hello, thank you for these great tools! I am using Akita for HiC prediction. I am getting the following issue during training:

I am using HiC data from ENCODE-- since the data is not binned at 2048 bp resolution, I used juicer and cooltoolsto generate a .cool file at 2048 bp resolution from the contact matrix.

I followed the preprocessing steps in tutorial.ipynb with the following parameters (exact same as from tutorial):

akita_data.py \ -l 1048576 \ --crop 65536 \ --local \ -o data_maleheart \ --as_obsexp \ -p 8 \ -t 0.1 \ -v 0.1 \ -w 2048 \ --snap 2048 \ --stride_train 262144 \ --stride_test 32768 \ data/hg38.ml.fa \ data/heart_cools.txt

I changed the params of the model to have params['model']['head_hic'][-1]['units'] = 1

and then, to train:

python akita_train.py -k -o ./data_maleheart/train_out ./data/test_model/params_tutorial.json data_maleheart

It appears that the targets are somehow mathematically incompatible with the model but I would greatly appreciate any pointers on how to debug this. Thank you so much again for your time and help.

[gfudenberg]

Hi Ananya, its often helpful to do a model print to look at the various layer sizes to make sure pooling agrees. Also, -t is just for testing!