1
$\begingroup$

I want a model to predict the probability of an event occurring. I would like to use logistic regression for this. An external condition that needs to be satisfied, however, to allow me to use such a model, is that the AUC is at least 70% on the test sample.

I was therefore wondering about the following:

  • if the model obtained through including all features in the logistic regression model yields an AUC below 70% on the test sample, will all more simple models (in terms of number of features included) also have an AUC below 70% on the test sample?
  • does the AUC of the model on the test sample obtained through including all features give an indication of the AUC upper bound on the test sample of all models containing less features?

If this is true, I do not need to look further if the model including all features has an AUC lower than 70%.

$\endgroup$
0
$\begingroup$

The short answer is no, sorry. If you include too many variables, then you are overfitting to your training data, and will possibly (probably) lose accuracy when you apply your model to a new data set. This is basically the whole reason that model specification is such a hard thing.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.