I am fitting a logistic regression model on a data set with about 200,000 observation and 100 features. According to SAS output, the model converged correctly with an in-sample AUC of 0.85. However, when I applied the model on a few out of sample data set, the AUCs are as high as 0.98. This doesn't seem right to me. Usually the in-sample AUC is better than out-of-sample AUC. Any thoughts on the reason why this happened and how to interpret such results? I was using proc hplogistic in SAS 9.4.
Thanks everyone for your opinion. To be more specific, I was fitting the model using data in 2013, while I was testing the model on the data for 2014 Jan-June. Would that be causing this and how to adjust for that if needed? Besides, for response variable, the ratio of positive and negative is about 1:1.