I'm using glmnet for prediction of case/control, which I created with the function train with additional parameters for cross validation (CV) and optimize sensitivity. I used the ROC values for comparisons.

If I add features that had a very good performance in a big model to our data, the specificity decreases by around 20%. It was a 100-fold CV for the robustness, and I tried it also with another seed.

How could it be that the specificity decreases if I'm adding good features?

I also used the ridge approach for regularization, because I read if there are correlated features in the dataset, the LASSO would randomly choose one of these. The specificity decreased again with ridge. The highest correlation is 0.65.

The code I used is as follows:

cvCtrl <- trainControl(method="repeatedcv", repeats=runs, number=10, 
                       summaryFunction=twoClassSummary, classProbs=TRUE)

    mod <- train(traindat, bin.fac[-idx], method=m, metric="Sens", 
                 trControl=cvCtrl, tuneLength=30, family="binomial", dfmax=n)
  • 2
    $\begingroup$ Is your question "why did the specificity decrease when adding good features?"? $\endgroup$ – Edgar Jun 6 '19 at 10:51
  • $\begingroup$ Hey Edgar, yes that´s my question. Do you have any suggestions? $\endgroup$ – user250106 Jun 13 '19 at 13:18
  • $\begingroup$ But the sensitivity is increasing if you add these features? $\endgroup$ – Edgar Jun 13 '19 at 15:33
  • $\begingroup$ I always look at a sensitivity value of 97 or 91 and if I combare it with the bigger model (with adding the good features) the specificity decreased. $\endgroup$ – user250106 Jun 14 '19 at 10:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.