0
$\begingroup$

In the paper UNITN: Training Deep Convolutional Neural Network for Twitter Sentiment Classification, as the name suggests, a CNN architecture for sentiment classification is being introduced:

enter image description here

I am currently trying to reproduce these results but I am new to CNN hence I don't really understand a few basic things here. My main question is about those different filters F and how they actually look like.

In image recognition such filters may be just some matrices for edge detection etc. but here we are working with word embeddings. Unfortunately the paper does not explain how they came up with their filters. One thing I could imagine is e.g. to first learn an auto encoder and use the weights of the encoder as a filter. However, as you can see from the picture there are multiple filters being used which does not support this theory because why would there be multiple filters then?

Hopefully somebody can shed some light into this.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

It works exactly like image recognition. These filters are learned through back propagation in the CNN. On the image you show, you see that several filters are learned resulting of a convolution on the sequence of words (here 3 words per sequence). At the end of the training, these filters can detect specific interesting patterns.

If you understand how CNN work for images, then you just have to know that these filters slides only on the columns of your input matrix, instead of squares in the case of images. The rest of the CNN works the same.

The tricky part of CNN for text analysis resides in the embedding dimension selection and the maximum number of words of columns in the sentence matrix.

Improve this answer
$\endgroup$
3
  • $\begingroup$ So, does that mean that I train the whole network, from the fully connected output layer back to the convolutional layer using backpropagation? That would mean that I also learn the weights from the convolutional feature map to the pooling layer and also the weights from the pooling layer to the softmax layer - is that correct? $\endgroup$
    – Stefan Falk
    Jan 13, 2017 at 13:28
  • $\begingroup$ Yes that's correct. I don't understand what you mean by "weights from the convolutional feature map to the pooling layer". I think that the pooling layers are just some kind of max function (not having weights). But then your softmax layer is definitely also learned by back-propagation. $\endgroup$
    – Robin
    Jan 16, 2017 at 8:40
  • $\begingroup$ @StefanFalk If this answers your question, can you mark it as accepted please? If not, please can you explain what information you are missing :) $\endgroup$
    – Hugh Perkins
    Mar 24, 2018 at 23:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.