NLP Emotion Detection - Model fails to learn to recognize negations

Question

I am working on a nlp emotion detection project. The emotions that I try to predict are 'joy', 'fear', 'anger', 'sadness'. I used some publicly available labeled datasets to train my model e.g. ISEAR, WASSA etc. I have tried the following approaches:

1. Traditional ML approached using bigrams and trigrams.
2. CNN with the following architecture: (X) Text -> Embedding (W2V pretrained on wikipedia articles) -> Deep Network (CNN 1D) -> Fully connected (Dense) -> Output Layer (Softmax) -> Emotion class (Y)
3. LSTM with the following architecture: (X) Text -> Embedding (W2V pretrained on wikipedia articles) -> Deep Network (LSTM/GRU) -> Fully connected (Dense) -> Output Layer (Softmax) -> Emotion class (Y)

The NN models achieve more than 80% accuracy but still when I use the the trained model to predict the emotion on text that includes some negation I get the wrong results. For example:

Text : "I am happy with easy jet, it is a great company!"

Predicts Happy

Text: I am not happy with easyjet #unhappy_customer

Predicts Happy

Any suggestions on how to overcome this problem?

Answer 1

Negation is a tough problem in NLP. If you search on Google Scholar for negation in sentiment analysis there are dozens of papers that suggest some solutions.

The relatively simple things you can is:

• Detect such cases in your training data and upsample them.

• Generate synthetic training data by negating some of the training sentences that are simple enough for some rule-based editing after you do some NLP analysis over them (I would do it in spaCy).