1
$\begingroup$

I have performed two ridge logistic regressions in R to check which of the two models perform better. From the first look of the coefficients, it looks like model1 has more influential variables than model2 - simply that the coefficient are closer to 0 in model2.

         model1      model2
Int  -2.04×10+1     1.86×10+1
V1   -2.14×10-1     8.08×10-3
V2    3.05×10-3     1.20×10-2
V3   -1.80×10-4    -2.22×10-4
V4   -4.33×10-4    -1.40×10-2
V5   -3.11×10-3    -9.87×10-3
V6    5.51×10-1     4.31×10-2
V7    2.94×10-2     2.42×10-3
V8   -4.04×10-1

Once I look at the model evaluation, all of a sudden model1 has no predictive power, while model2 shows relatively good predictive power.

auc

The overall script is found here

Why does the individual variables for model1 seem more influential, but evaluated all together have no predictive power, and vice versa for model2?

$\endgroup$

1 Answer 1

0
$\begingroup$

You are mixing two different concepts which are not necessarily related. Just because a model has (more) non-zero coefficients than another (what you call "influential"), this does not mean that it should predict better. In fact, that's the whole purpose of penalized regression, to find a set of shrinked coefficients which predicts better. Sometimes your problem may be very noisy, in that case having most coefficients at zero is probably better, as they are likely to be modelling noise.

In conclusion, looking at the coefficients tells you nothing about the model's predictive power.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.