Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have a few parameters that are related (let's call them X1 and X2), and I want to use whichever one will provide the strongest model. The model has many other parameters. Would I simply be able to compare the AICc of these two models?

Model using X1:

Y ~ X1 + X3 + X4 + X5 + X6 

Model using X2:

Y ~ X2 + X3 + X4 + X5 + X6

This is confusing me as I'm not sure what other parameters will ultimately belong in the "correct" model. Ultimately, X3, X4, etc. may be thrown out. So would I want to test this with potentially "bad" parameters?

share|improve this question
1  
What does "strongest" mean in terms of models? Having parameters that don't contribute much explanatory power in your model doesn't usually cause much harm (some inflation of standard errors, when there's multicollinearity may be an issue) - but removing predictors that don't achieve significance certainly can cause problems. –  Glen_b Aug 31 at 23:14
    
This is a chapter in a regression textbook. The people here are fairly good a condensing information so you might get something in one of the answers, but I think you'd be better served by revisiting a textbook. (why are you using AICc, how many outcomes do you have, why is this a mixed-model, what is the goal of model building??) –  charles Sep 1 at 0:01

1 Answer 1

up vote 1 down vote accepted

As long as the data you are using is the same, you can compare the AICc of different models (I suggest you follow Burnham and Anderson and use AICc always). When you "throw out" a parameter, you will get a different value, as both the likelihood and the "parameter penalty" (in AICc, per B&A, it isn't exactly a penalty, but we'll abuse the notation here) will change. If you test with the "bad" parameters, you should see the AICc increase, as any decrease in the likelihood (and it may be an increase if the parameter is really bad) will be offset by the penalty component. The model with the smallest AICc value is your "strongest" model, but models whose AICc is within 2 of the minimal AICc are usually considered of equivalent strength.

share|improve this answer
    
Thanks for your answer. My main concern is running too many models without an a priori game plan. Do you think I should test the "full model" with each parameter and compare those two models or do a step-wise method of some sort with each parameter and compare the two "best" models. –  user14241 Aug 31 at 23:26
    
My gut feeling is to just compare the "best" models: the best X1 with the best X2. After all, you aren't going to use the full model if a reduced one is better, will you? –  Avraham Aug 31 at 23:46
    
This sounds very reasonable to my novice mind. I don't have the reputation to vote this up, but hopefully I will soon, and I will come back and do it. I appreciate your help. –  user14241 Sep 1 at 0:09
    
Thanks! I will do that right now. –  user14241 Sep 1 at 0:31

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.