1
$\begingroup$

I tried to ask in SO, but they told me to ask here.

I have a big dataset like this:

RELATION<tab>SENTENCE
color<tab>The cat is black
color<tab>My dog is white
place<tab>Des Moines is in Iowa
place<tab>Des Moines is the capital of Iowa
is-a<tab>Des Moines is a city
is-a<tab>3D printer is a type of printer
is-a<tab>New Beetle was a car by Volkswagen
...

I need to build a classifier that, given a sentence, returns a relation as accurately as possible.

I have already programmed something with keras (python), but in this case I really don't know where to start from. For the moment I only realized that an important feature could be the order of the words in the sentence, but I don't know how to explot this.

Do you have some hint? E.g. about features, embeddings, hidden layers. May LSTM be a good NN? Why?

I hope this is not too broad, but I just need some hint.

$\endgroup$
0
$\begingroup$

A good start would be using some pre-trained embedding like word2vec to transform the sentences into lists of vectors. As opposed to other feature representations for text classification (e.g., TF-IDF) here the structure of the sentence is taken into account. So the first layer of your network could be this word2vec-based embedding; then, you can put some Conv1D/pooling layers, to end with a couple of dense layers and a softmax final layer. Of course, you will have to zero-pad some of the samples to make them all have the same length. You could also use LSTM to manage the sequential nature of words in sentences. However, in my experience if your texts are relatively short and there are not huge length differences, the Conv1D/pooling combo will work well and will be easier to train.

Your final architecture could look something like this:

enter image description here

We used that kind of network to solve a similar problem in a recent paper [1]. However, in this case the final layer wasn't using softmax, but a simple dense layer trained to perform metric learning [2].

[1] https://link.springer.com/article/10.1007/s10489-017-1109-7

[2] https://cs.nyu.edu/~sumit/research/assets/cvpr06.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.