I have a huge dataset, say around 100M data points, with a class imbalance of 1 positive for every 100 negatives. It is very difficult to train on the entire dataset, so I tend to undersample the negatives such that the training data becomes balanced (1:1). But the test set remains imbalanced to reflect the real life nature of the data. FYI, I use a simple feedforward neural network
How do I go about training in such a combination? I'd use class weights during training but I'm thinking it might overpredict on the test set? Moreover how do I evaluate this model with AUC and AUPRC, do I need to use some form of weighting?