I am trying to implement a LSTM based classifier to recognize speech. I have a dataset of speech samples which contain spoken utterences of numbers from 0 to 9. Each file contains only one number. I have extracted 13 mfcc and each file contain 99 frames. Therefore I have (99 * 13) shaped matrices for each sound file.

x_train => (100, 99, 13) => [[frame1 frame2 .... frame99],[frame1 frame2 .... frame99]]
y_train => (100,) => [0 0 0 0 ....... 9 9 9 9]

Following the examples mnist_irnn of keras I have implemented my network as follows. (I just set the hidden units to 13 because I wanted to make it fast)

from __future__ import print_function
import numpy as np

from keras.optimizers import SGD
from keras.utils import np_utils
from keras.models import Sequential
from keras.layers.core import Dense, Activation
from keras.layers.recurrent import LSTM
from SpeechResearch import loadData

'exception_verbosity = high'
batch_size = 5
hidden_units = 13
nb_classes = 10
print('Loading data...')
(X_train, y_train), (X_test, y_test) = loadData.load_mfcc(10, 2)

print(len(X_train), 'train sequences')
print(len(X_test), 'test sequences')
print('X_train shape:', X_train.shape)
print('X_test shape:', X_test.shape)
print('y_train shape:', y_train.shape)
print('y_test shape:', y_test.shape)
print('Build model...')

Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)

model = Sequential()
model.add(LSTM(output_dim=hidden_units, init='uniform', inner_init='uniform',
               forget_bias_init='one', activation='tanh', inner_activation='sigmoid', input_shape=X_train.shape[1:]))


sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd)

model.fit(X_train, y_train, batch_size=batch_size, nb_epoch=3, validation_data=(X_test, y_test), show_accuracy=True)
score, acc = model.evaluate(X_test, y_test,
print('Test score:', score)
print('Test accuracy:', acc)

I have tried my best to set the values according to the example. But when I run the code I am getting following error. (I will include print statements as well because that help to get an idea about dimensions about each matrix)

Using Theano backend.
Loading data...
100 train sequences
20 test sequences
X_train shape: (100, 99, 13)
X_test shape: (20, 99, 13)
y_train shape: (100,)
y_test shape: (20,)
[0 0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9]
Build model...
Train on 100 samples, validate on 20 samples
Epoch 1/3
Traceback (most recent call last):
  File "/home/udani/PycharmProjects/testResearch/SpeechResearch/lstmNetwork.py", line 58, in <module>
    model.fit(X_train, y_train, batch_size=batch_size, nb_epoch=3, validation_data=(X_test, y_test), show_accuracy=True)
  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 581, in fit
    shuffle=shuffle, metrics=metrics)
  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 239, in _fit
    outs = f(ins_batch)
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 365, in __call__
    return self.function(*inputs)
  File "/home/udani/Documents/ResearchSW/Theano/theano/compile/function_module.py", line 871, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/home/udani/Documents/ResearchSW/Theano/theano/gof/link.py", line 314, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/home/udani/Documents/ResearchSW/Theano/theano/compile/function_module.py", line 859, in __call__
    outputs = self.fn()
ValueError: Input dimension mis-match. (input[0].shape[1] = 1, input[1].shape[1] = 10)
Apply node that caused the error: Elemwise{Composite{(i0 * log(i1))}}(<TensorType(float32, matrix)>, Elemwise{Clip}[(0, 0)].0)
Toposort index: 215
Inputs types: [TensorType(float32, matrix), TensorType(float32, matrix)]
Inputs shapes: [(5, 1), (5, 10)]
Inputs strides: [(4, 4), (40, 4)]
Inputs values: [array([[ 0.],
       [ 9.],
       [ 8.],
       [ 8.],
       [ 6.]], dtype=float32), 'not shown']
Outputs clients: [[Sum{axis=[1], acc_dtype=float64}(Elemwise{Composite{(i0 * log(i1))}}.0)]]

HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

Process finished with exit code 1

I am unable to figure out why am I getting this exception. I am really thankful to anyone who can help me to solve this. I am totally new to these network modeling and Keras.


1 Answer 1


As I can see there are some mistakes in your code.

model.fit(X_train, y_train, batch_size=batch_size, nb_epoch=3, validation_data=(X_test, y_test), show_accuracy=True)
score, acc = model.evaluate(X_test, y_test,

In above code you should change following parts, y_train to Y_train and y_test to Y_test

Then your code will work fine! good luck!

  • $\begingroup$ Ya it worked. I have been using y_train and y_test instead of using the new variable. My mistake.... :) $\endgroup$
    – udani
    Commented Jan 22, 2016 at 18:36

Not the answer you're looking for? Browse other questions tagged or ask your own question.