In computer vision, CNNs have definitely proven themselves useful in learning feature maps for tasks such as image classification. When applied to images, the role of convolutional and max-pooling layers is easily interpretable as image filtering and aggregation. However, what does it mean to apply CNNs for textual data? What is the interpretation of CNN layers in this case?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Instead of image pixels, the input to most NLP tasks are sentences or documents represented as a matrix. Each row of the matrix corresponds to one token, typically represented as a word embedding like word2vec or GloVe.
In vision, convolutional filters slide over local patches of an image, but in NLP, filters are typically used over full rows of the matrix. Thus, the width of the filters is usually the same as the width of the input matrix. The height, or region size, may vary, but sliding windows over 2-5 words at a time are typical.
A big argument for CNNs is that they are fast. The most natural fit for CNNs seem to be classifications tasks, such as sentiment analysis or spam detection. Convolutions and pooling operations lose information about the local order of words, so for example sequence tagging is harder to fit into a pure CNN architecture.