1
$\begingroup$

I'm doing logistic regression in R with binary data (0's and 1's), sample size around 300 : Predicting 1 target variable (varp)

If I use one independent variable ( varx), it's significant (p 0.03, the AIC is 200) : glm(formula = varp ~ varx, family = binomial, data = mydata)

 Coefficients:
                Estimate Std. Error z value   Pr(>|z|)    
    (Intercept)  -1.0251     0.2215  -4.681 0.00000245 ***
    indepvar1  -0.6551     0.3612  -2.118     0.0322 *  
Dispersion parameter for binomial family taken to be 1)
    Null deviance: 211.06  on 205  degrees of freedom
Residual deviance: 206.36  on 204  degrees of freedom
AIC: 200

But when I use multiple independent variables of interest the AIC becomes 170, glm(formula = varp ~ varx+varb+vargg+varkkk...., family = binomial, data = mydata)

How to select the model ( the one with 1 var or the group of vars) that best predict the varp ?:

  1. the model with One independent variable (varx) with AIC 200 , or
  2. a group of variables with AIC 170, in this group, the varx becomes non significant and instead another one is significant ...
Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

First the answer: Well, based on your short explanation I would say 2, if you want to predict the dependent variable. If your goal is to model parsimony, then use AIC, if predictive power then adjusted R2. Notice, the adjusted as we in regular regression tend to look at adjusted R2 rather than just R2. You can maximize the predictive power of your model by evaluating prediction error metrics (MAE, RMSE, etc). And maybe don't compare AIC to R2, compare AIC with the change in adjusted R2 instead.

Second some food for thoughts: I do not get the reasoning for your choice of models. Why do you run a model with only one independent variable when you are in possession of more potentially descriptive variables? That does not make any sense in my opinion. Include all the variables in your predictive model instead and you do not have to compare it to anything else.

If it is because you want to know which variables to include or not you should look more into the tag feature-selection and maybe the glmnet package in which you can diminish insignificant independent variables to 0 and then get the feature selection.

Improve this answer
$\endgroup$
14
  • $\begingroup$ Q:> do not get the reasoning for your choice of models. Why do you run a model with only one independent variable when you are in possession of more potentially descriptive variables? A: Becuase when I use other descriptive variables, only one from 10 are significant..., the remaining 9 are not significant with that AIC that seems better... $\endgroup$
    – Den
    Aug 20, 2020 at 14:29
  • $\begingroup$ Btw. I adjusted my answer. I meant go with 2, where AIC is smaller and variables are included. I would really recommend you to read more into the discard of non-significant variables. It is seen as a bad commonly adopted idea in statistics, here. And, I still think a model with only one variable is not really a model, at most a very very dubious model. Just because 9 of the 10 variables are not significant does not tell you they are useless. $\endgroup$
    – Thomas
    Aug 20, 2020 at 21:50
  • $\begingroup$ Cont'd: And, are you sure you then have all the descriptive variables for your independent variable? Maybe you miss some! Try have a look at the glmnet package and the opportunities with ridge, lasso & elasticnet models for binary outcome here $\endgroup$
    – Thomas
    Aug 20, 2020 at 21:50
  • $\begingroup$ 1, 3 or all 10? if we speak about health data, getting all health variables is useless, if someone have a rib fracture and there is a variable 'tooth_pain'? so why including it .? it's the point, I'd like to narrow that infomatoin and since it's dichotomic data I don't see better things than logistic regression. .... I'm getting confused with the 10 variables now, since I thoght less are better and more explanatory if working with industry specific data. do yo use 'Targets' or other package that can deal with that situation in an automatic matter. Any examples using lasso / glmnet ? $\endgroup$
    – Den
    Aug 21, 2020 at 9:21
  • $\begingroup$ Of course, you also have to have a certain constrain in picking out your variables. But, in your case, it just seems like (I still have no idea what data you are actually using, as it has not been provided) that the 10 variables chosen are not all so good predictors for your binary outcome, or maybe they are just better predicters for 0. I am not talking about including unintuitive variables but just questioning if you really catch it all with the 10 variables. Where did you read less is better than more? $\endgroup$
    – Thomas
    Aug 21, 2020 at 11:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.