Abnormal behavior while predicting with training data

Question

I am trying to predict the appearance count of a particular item using Neural Network.My data has severe imbalance i.e 90% of data contains zero as output, remaining values ranges from 1 to 5. I am training a neural network with item features and week number of year.Whenever I run the neural network and **predict for the training data the sum of all predicted counts are equal to the sum of observed counts but the individual counts are not even getting close.I tried with both RMSE and MAE as error metrics.**I am not sure about this is correct, any help would be great

Answer 1

The problem with Machine Learning classification models is that they assume the input data have almost same number of data for each class. I had a dataset having classes in the ratio 95% and 5%. So, of course when I did prediction, I got 100% accuracy (which you can never get on real world data). To remove this problem, I used under-sampling of data (took approx same number of training data of majority class as was minority class). Of course this is not the ideal method, but it worked for me.

You can do the same with your data. I suggest you to look at this- https://www.analyticsvidhya.com/blog/2017/03/imbalanced-classification-problem/

https://elitedatascience.com/imbalanced-classes

Many links will give you the following methods to tackle this problem - Over-sampling of minority class, under-sampling of majority class, using tree based methods etc. Its up to you to decide, take the one which gives you better output.