3
$\begingroup$

I am trying to do binary classification of news articles using Recurrent Neural Net with word embedding. Following are the parameters of the model:

Data:
    8000 labelled news articles (Sports:Non-sports::15:85)

Parameters:
    embedding size = 128
    vocabulary size = 100000
    No. of LSTM cell in each layer = 128
    No. of hidden layers = 2
    batch size = 16
    epochs = 10000

Result:
    AUC on training set = 0.60
    AUC on testing set = 0.55

As the both training and testing error is high model is underfitting and require more data. So I have couple of doubts here:

  1. What would be the optimum data size required?
  2. Can we change the parameters to improve AUC. By decreasing, embedding size or No. of neurons we can minimize degree of freedom.
$\endgroup$
1
$\begingroup$
  1. What would be the optimum data size required?

The more the better. From my experience in text classification, 8000 samples should yield good results in most cases. AUC on testing set = 0.55 means your classifier is almost random: I would expect better.

See How to get the data set size required for neural network training? or How few training examples is too few when training a neural network?.

  1. Can we change the parameters to improve AUC. By decreasing, embedding size or No. of neurons we can minimize degree of freedom.

You have to experiment. Your hyperparameters look reasonable to me, but you may want to try CNN as CNN tend to outperform RNN for text classification when texts are not short (> 100 words).

I did some experiments on (sequential) short-text classification:

FYI:

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.