I ran logistic regression on a data of 3700 patients. I have 9 variables and my outcome is presence of a disease or not. I got the regression coefficients and predicted probabilities. When I apply this model on another data set, no matter what I do the area under ROC curve does not go above 56%.

I am assuming there is underfitting in my model. How can I improve this and reduce the high bias? Any way to calculate the bias in a software? How can I fix this underfit in a software?

Thank you very much to anyone who provides a solution.

  • $\begingroup$ What was the AUC for the original model. This sounds more like overfitting than underfitting, but I can't tell from what you've put. $\endgroup$ Nov 15 '16 at 16:39
  • $\begingroup$ Since the original data was used to generate the model, AUC is 99.9% $\endgroup$ Nov 16 '16 at 13:53

From what you described, it is hard for me to say it is under-fitting. It is even possible over-fitting. I would suggest to use "learning curve" plot to check the problem.

How to know if a learning curve from SVM model suffers from bias or variance?

Suppose you verified it is under-fitting. Basis expansion can be used to increase the variance of the model. The basis expansion can be polynomial expansion or spline expansion. Details and examples can be found in

Why are there large coefficents for higher-order polynomial

  • $\begingroup$ Thank you very much Sir. Can we still apply learning curves for training and test data when the sample size is different for both? $\endgroup$ Nov 16 '16 at 13:17
  • $\begingroup$ @Faiz_Yusufi Yes. I think training and testing does not need to be have same size. $\endgroup$
    – Haitao Du
    Nov 16 '16 at 14:05
  • $\begingroup$ Ok Sir. I will learn the learning curve and perform it on my data. Can I please have some way to contact you to ask for more assistance and inform about my results? $\endgroup$ Nov 17 '16 at 19:44
  • $\begingroup$ @Faiz_Yusufi i do not think that is the way Stack Exchange works. People may busy for other issues and may not want to leave contact info. $\endgroup$
    – Haitao Du
    Nov 17 '16 at 19:46
  • $\begingroup$ No problem Sir. I will work on my problem and update the progress here. Thank you so much. $\endgroup$ Nov 17 '16 at 20:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.