I'm currently attempting to make a Seq2Seq Chatbot with LSTMs. The data I used is from Cornell's Movie Dialog Corpus.

Here's the link to my code on GitHub, I would appreciate it if you took a look at it: Seq2Seq Chatbot You need to change the path of the file in order for it to run correctly.

I'm using 2 GTX 1080 with 8GB RAM, and I'm training my code with GPU support.

Here's the error I got:

ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[3,1143,44592]
         [[Node: training/RMSprop/gradients/dense_1/Max_grad/Cast = Cast[DstT=DT_FLOAT, SrcT=DT_BOOL, _class=["loc:@dense_1/Max"], _device="/job:localhost/replica:0/task:0/device:GPU:0"](training/RMSprop/gradients/dense_1/Max_grad/Equal)]]
         [[Node: loss/mul/_93 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_3108_loss/mul", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Exception ignored in: <bound method Session.__del__ of <tensorflow.python.client.session.Session object at 0x7f18568a7e10>>
Traceback (most recent call last):
  File ".local/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 696, in __del__
  File ".local/lib/python3.4/site-packages/tensorflow/python/framework/c_api_util.py", line 30, in __init__
TypeError: 'NoneType' object is not callable

It's telling me that I ran out of memory.

The funny thing is, I'm currently dividing my input sentences into batches of 20(I changed this value, description further down below) and training them, and I see that the program is dividing a lot of chunks of the memory(i.e., Something similar to 2018-01-14 17:09:40.102649: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 1002348032 totalling 955.91MiBis printed to the console a ridiculous number of times) for just this one batch of data, and the program never gets to the next batch.

Following similar issues to mine I saw on the internet,I've tried:

  • Reducing memory usage by changing the data type: float32 -> float16
  • Reduced the batch size to 10, 5 and then 3
  • Reduced the epochs to 3

and none of these worked. The majority of the similar issues I saw were all using image data, by the way.

I'm thinking it may be related to the size of one batch(it's a numpy array of [3,1143,44592]), or simply an error in my code; but honestly, I'm terribly stuck right now.

Any help would be greatly appreciated!


2 Answers 2


This question is better asked on StackOverflow, but I'll give you a hint.

First up, the tensor that the engine is trying to allocate is enourmous:

$$1143\cdot44592\cdot3=152905968 \approx 150M$$

With float32 variables, it takes ~600Mb, but even with float16 is't ~300Mb, which is a lot. And this is just one layer of the network (not including the training data and intermediate tensors and internal tensorflow infrastructure). Also note that the RMSProp optimizer actually doubles all variables to perform gradient updates, so the memory is doubled as well.

I've skimmed through your code and noticed that you use such a big layer for softmax output for word prediction. Not only is it inefficient in terms of memory, this approach is also hard to train.

In NLP, it's common to use sampling loss function in order to classify words in a large vocabulary, most commonly negative sampling and NCE. These losses are implemented in tensorflow, but require a bit of manual work in keras (see this discussion on GitHub), but they are much more memory and computationally efficient.

So I think the biggest improvement for you would be to implement NCE loss function. You can also try to train with plain sgd optimizer to save memory.

  • 2
    $\begingroup$ re your first line: Migration to Stack Overflow was rejected because the OP is blocked from asking questions there. $\endgroup$
    – whuber
    Commented Jan 16, 2018 at 19:15
  • $\begingroup$ Ok. Thanks. Just curious if it's ok to ask programming questions on CV in this case? $\endgroup$
    – Maxim
    Commented Jan 16, 2018 at 19:16
  • 2
    $\begingroup$ That's viewed as circumventing SE policies. Such efforts are usually greeted with substantial negative prejudice :-). $\endgroup$
    – whuber
    Commented Jan 16, 2018 at 19:18

You don't quiet have your LSTM setup right. Some thoughts:

  1. Use keras.preprocessing.text.text_to_word_sequence to turn your texts into sequences of word ids.
  2. Use keras.preprocessing.sequence.pad_sequences to truncate/pad all your sequences to something like 32 or 64 words.
  3. Use an embedding layer after your input layer to map the sequences of word ids to a sequence of word vectors.
  4. Use the sequences of word vectors as input to to the LSTM
  5. Try a GRU instead of an LSTM (it's a little bit simpler)
  6. After the decoding layer, the output will be a sequence of word vectors. Use nearest neighbors to lookup the nearest word in your vocabulary for each "word" in the output.

You can learn the embeddings while training the network, or you can use a pre-trained embedding layer.


Not the answer you're looking for? Browse other questions tagged or ask your own question.