This is not a problem:solution scenario insofar that I am not attempting to find a way to improve the model, merely find a reason for its behaviour.

Model using LSTM has accuracy of about 84-86% while LSTM featuring a CNN layer (CNN followed by LSTM) has accuracy of about 82% on the same dataset.

Received wisdom seems to suggest that adding a CNN layer would improve performance. The hyperparameters of the CNN was determined through Bayesian optimization, with quite a broad search space (kernel size, activation function, number of convolutional filters, etc.). 82% seems to be about the best it can do, with an ideal setup.

Are there are any reasons which might help explain the slight drop in performance when adding a convolution layer to a RNN? The data in this case are word vectors generated from a noisy source (think Twitter).

  • $\begingroup$ Can you guve more info about the two models? Number of layers in each, activations, etc. $\endgroup$ – SaTa Mar 16 at 2:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.