I have been working on a logistic regression model to predict 'yeses' in a yes/no classification problem. The objective of my problem is not necessarily to predict the outcome, but it's rather to just get a better understanding of my variables and how they influence the outcome.

For example, I want to say that feature X is 2.2 more likely to achieve 'yes' then my reference level, and Y feature Y is 2.5 less likely to achieve 'yes', etc.

I did my model, received the output, and I know how to read the coefficients, but I want to know if there are any tests I can do to confirm the confidence of the coefficients.

I did a few things already: 1. checked the P-value of the summary (focused on the ones <.05) 2. Ran each model predictor by predictor to look at the the change in prediction, BIC, AIC, AUC, confusion matrix, ROC curve, etc. 3. I tried to do a chisq test of independence on my independent variables (since they are all categorical), but I'm getting weird results with R saying the results may not be accurate and the result is p_value <.05. The contingency table do show some very low frequency so I'm wondering if that has to do with anything. 4. Did a partial dependency plot....which seems to show opposite direction of my coefficient, but that may be just a software problem on my part. 5. the original data is unbalanced, so I 'up-sampled' it in R. 6. Accuracy is about 70%, AUC is about 79%

Besides these things, are there anything else that I can do? Basically, if I say that a feature is 2.5x more likely to vote 'yes', I want to make sure that is more or less correct.

Right now, I feel like i"m just taking the output at face value and it's a bit uncomfortable.

  • $\begingroup$ Would features X and Y in your description be indicator variables? $\endgroup$ – Dave Nov 18 '19 at 4:36
  • $\begingroup$ so in my example, X could be Gender-Male, and Y could be Region-Texas. so Gender-Male is 2.2 more likely to say 'yes' and Texas is less likely to say yes. $\endgroup$ – semidevil Nov 18 '19 at 4:37
  • $\begingroup$ Okay, then your interpretation of the parameters makes sense. $\endgroup$ – Dave Nov 18 '19 at 4:39
  • $\begingroup$ @Dave: It doesn't seem to me that the interpretation of the estimated regression coefficients is correct. If the coefficients were exponentiated, they would compare odds of a yes (rather than a no) for males compared to females, say, all else being the same. There is a difference between comparing odds and probabilities! Whenever language like "is more likely" is used, that suggests the interpretation is done on the probability scale (which is incorrect) rather than the odds scale. $\endgroup$ – Isabella Ghement Nov 18 '19 at 5:05
  • $\begingroup$ “2.5 times more likely” sounds like odds to me. @semidevil what exactly do you mean? (“I’m not sure” is an acceptable answer.) $\endgroup$ – Dave Nov 18 '19 at 5:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.