Training is too slow on GPU (batch size = 1)

[nrc53]

Hi,

I am training the network on NVIDIA Tesla GPU with a batch size of 1. I am training for 500K iterations, it's taking huge time, around some 600 hours. I listed below the parameters I used.
batch_size = 1
sequence_max_length = 100
words_count = 256
word_size = 32
read_heads = 4
learning_rate = 1e-4
momentum = 0.9

Any suggestions on how to improve the training time?

Thanks