0
$\begingroup$

Logistic Regression using R's glm package is giving me the following summary (snap of the few first variables).

My Data Set:

  • Dimensions: 1252 rows and 224 columns (after using model.matrix). The Data has been standardized.
  • Response variable is binary.
  • Trying to predict if an employee will leave the company, based on employee attributes

enter image description here

My Understanding:

The model does not give a good fit because:

  1. Residual Deviance > Null Deviance.
  2. p.value = 1 - pchisq(3676.5, 817) turns out to be 0.
  3. The first warning about 0 or 1 fitted probability message suggests that due to some predictor(s) the model might be giving perfect predictions
  4. Second warning on ‘rank deficiency’ suggests that there might be predictors that are linearly dependent on one another.

My Questions:

  1. How can I improve the model? I would like to see Residual Deviance < Null Deviance. I will invest time on dropping the linearly dependent variables in the model, but is there anything I should do first to test the ‘model’ itself, before revisiting my data? I am asking this because SVM worked quite well on the same data set.
  2. Why do I have such extreme coefficient values?
  3. Many answers to other posts state that ‘AIC’ is used to compare different
  4. The summary parameters (coefficients , std error and p-values) for many dummy factors obtained via model.matrix, like GSS_SEXM, is shown as 'NA'. Why is it so?
  5. logistic models. What is meant by ‘different’ here? Models trained on different data sets that bear different coefficients, like say different set of attributes?
$\endgroup$
  • 2
    $\begingroup$ My first thought, seeing the coefficient size, is that something very bad happened. Oh, that’s what the message in red is telling us... Look for example at GSS_SEXM: of course you can’t estimate the effect of this variable together with GSS_SEXF, as GSS_SEXF+GSS_SEXM = 0. I think this is the case for many of your variables, you can sum some of them to obtain another one. This is not a well defined model. $\endgroup$ – Elvis Feb 26 '16 at 21:04
  • $\begingroup$ Thanks for your response. GSS_SEX was 1 column earlier, with 2 factors, "M" and "F" in it. I could not get 'glm' running without using 'model.matrix' which splits each of the factors into different columns. This is why I had to first split each such variable with multiple factors in it into separate columns. $\endgroup$ – batool Feb 26 '16 at 21:15
  • 2
    $\begingroup$ Welcome to our site! I edited a few things to improve the formatting (see our editing help for more information). By the way, on this site there's no need to say "thank you" at the end of your post - it might seem rude at first, but it's part of the philosophy of this site (tour) to "Ask questions, get answers, no distractions" and it means future readers of your question don't need to read through the pleasantries. I wonder if you might be able to make your title more specific, somehow, but I don't have a good suggestion right now. $\endgroup$ – Silverfish Feb 26 '16 at 21:16
  • 1
    $\begingroup$ I would guess that you have found perfect separation in your logistic regression, due to the large number of predictor variables for this number of cases. This is discussed extensively elsewhere on this site, for example here and here. Follow the hauck-donner-effect tag on this site. Please read such other posts and edit your question to focus on any aspects that aren't already covered. $\endgroup$ – EdM Feb 26 '16 at 21:38
  • 1
    $\begingroup$ @EdM I was able to fix the collinearity effect, as indicated by the fitted probability warning by using VIF function of 'rms' package. It told me around 50 variables were causing collinearity. After removing them, the warning did go away, but so did some of the variables that intuitively seemed important. Length of service was one such variable. $\endgroup$ – batool Mar 3 '16 at 17:49
1
$\begingroup$

As these data are based on employee records, you presumably have data on the time to quitting (length of employment), not just the fact of having quit. If so, this would be better modeled with survival analysis. Predicting the length of employment would seem to be of considerable value to the company.

Then the dependent variable is continuous, with those who haven't quit yet treated as "censored" observations. (We all do, eventually, end up leaving employment.)

Whether you model this as logistic or survival, you should carefully limit the number of variables under consideration or use a penalized method like LASSO or elastic net. The rule of thumb to avoid overfitting if you are not using a penalized method is to consider no more than one variable per 15 events. That would be the number who quit or otherwise left employment for survival analysis, or the smaller of those who quit/didn't quit for logistic (which, the more I think on it, seems less and less useful here). And in terms of the number of variables, each categorical variable counts as one less than the total number of categories (that's how many columns it contributes to the model matrix).

To make this concrete, say that 600 out of the 1252 cases represented people who left employment with the company. If you intend to do standard survival analysis, this rule of thumb means that you should enter no more than about 600/15=40 variables (columns of a model matrix) into your analysis, not the full model matrix with 224 columns. If only 300 people in your data set left employment, only 20 variables should be considered in standard survival analysis. The particular variables might best be selected based on your knowledge of the subject matter, or multiple correlated predictors might be combined into single predictors. If you need to evaluate more predictors than warranted by this rule of thumb you should use a penalized method.

$\endgroup$
  • $\begingroup$ Thanks for the excellent direction towards survival analysis. Could you please elaborate a little more on what do you mean by 'if half quit, no more than 40 variables'. Just to add here, my output variable for logistic regression was if a person left in a 'particular year'. $\endgroup$ – batool Mar 3 '16 at 20:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.