4
$\begingroup$

I want to choose variables for building a logistic regression model by comparing the GOF of different types. The problem is that some of the candidates have some missing values so I can´t use the likelihood ratio test for comparing models, given that when I want to compare models with those variables they are not nested anymore.

So after exploring some options, I found that AIC or even BIC might be an option for that purpose. Is either of those a valid option?

I am not familiar with the use of BIC, is it the same as AIC, where the smaller the value is, the better?

Improve this question
$\endgroup$
1
  • 3
    $\begingroup$ I'm not sure I understand why missing data means your models are not nested. Is it because cases are getting deleted if they don't have values for predictors in one model but not getting deleted in the more minimal models? That would pose a problem for AIC/BIC too, because they only provide meaningful comparisons when used on data sets with the exact same set of observations/cases. $\endgroup$
    – Anne Z.
    Commented Feb 8, 2012 at 15:08

3 Answers 3

Reset to default
3
$\begingroup$

Two points. First, I think it's a good idea to keep the sample the same across all models that you are comparing. That way it's a not an oranges to orangutans comparison. Maybe that puts you back in the nested world, and that will allow you to use conventional hypothesis testing. Second, the criterion you want to use depends on what you plan to do with the model. The trade-off between complexity and parsimony usually depends on the application. See my post on the AIC for more details.

Improve this answer
$\endgroup$
1
$\begingroup$

You can use AIC to compare them. The BIC is appropriate if you're doing a Bayesian analysis or it better captures what you theorize to be a good or useful information criterion. If you don't know what any of that is then use AIC. Some have recommended using BIC to assist in deciding marginal AIC values.

A lower value is better in both (smaller implies closer to 0?). Usually the cutoff for deciding better vs. inconclusive is about a difference of 6. If you need to report values of AIC report delta values because the absolute value of the AIC is relatively meaningless. In some measures there is some qualitative, and perhaps quantitative, difference between finding values of 1 and 9, and 10001, and 10009. This isn't so for AIC and the absolute measures just confuse matters.

Improve this answer
$\endgroup$
0
$\begingroup$

The BIC is justified from the Bayesian viewpoint but you din't need to be taking a Bayesian approach to use it. It is very similar to AIC. The difference is in the form of the penalty function on the number of parameterrs used. AIC was introduced by Akaike in the mid 1970s. BIC was introduced by Gideon Schwarz in 1978.

Improve this answer
$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.