I am using glm (target, formula = target~.,family=binomial) to predict binary outcome.

I have 9 grouped predictors. I convert them into factors so that I can test them in the regression as dummies.

Additionally, I have initially set the reference group relevel=var_name(var_name, ref = "ref_group_name")

I run the regression and the result is decent at first glance (good Gini, just a few dummies with high significance).

What I would like to do further is as follows:

1) check Mallows' Cp ( one thing to be checked for overfitting of the model that I know is that Mallows' Cp should be close to the number of dummies entering the model)

2) interactively add/remove dummies in the model

3) check correlation between dummies rather than between the whole variables

For the first question I am not sure how to test Mallows' Cp (summary of the glm result does not show it)?

For the second question I found package called dummies(dummy.data.frame function) that can easily convert factor variables into dummies so that I can interactively add or remove them (I found that potentially I can do that with package leaps(update function)) What I am missing is when I create the dummies data frame with dummy.data.frame how can I select which is the reference group when I put the resulting data.frame into the glm function?

For the third question: does it now make sense to run Pearson/Spearman correlation on the dummies data.frame (note: initially variables are both numeric and categorical before they were grouped)

  • $\begingroup$ Some but not quite all of this seems to be focused on how to do things in R. Please see advice on software-related questions in the Help Center. I am not voting to close given the remainder. $\endgroup$ – Nick Cox Jun 15 '15 at 9:13
  • 1
    $\begingroup$ $C_p$ was the joint invention of Cuthbert Daniel and Colin Mallows. Mallows later was the one to publish, so he is remembered. Either way the possessive is Mallows' or Mallows's, but definitely not Mallow's. (This is one of the most common typos in statistics.) $\endgroup$ – Nick Cox Jun 15 '15 at 9:15
  • $\begingroup$ LOVRIC 2011 "International Encyclopedia of Statistical Science". p.321. The Cp statistic is often used to guide selection of a subset model, but this cannot be recommended; while for each P separately, Cp gives an unbiased estimate of the scaled mean-square error for that subset, this is not true if the subset is chosen to minimise Cp. In fact this approach can lead to worse results than are obtained by simply fitting all available regressors. In a 1995 paper, Mallows has attempted to quantify this effect. $\endgroup$ – Erdogan CEVHER Jan 1 '17 at 10:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.