0
$\begingroup$

I have done a hierarchical regression (where the first model have 1 predictor, the second model have 2 predictors and the third model have three predictiors)and just the first model have a significant p-value (p<0.05). Should I choose the first one based on it's significance and exclude the others or should I look to other things* in the model in order to choose the best one?

*other things like the AIC or run anova test to compare the three models, etc..

$\endgroup$
0
$\begingroup$

I think it depends on your design. If you are doing an exploratory analysis, and building up the model complexity to see if the addition of predictors improves the model fit, then I would use the Likelihood Ratio Test to determine which variables improve the goodness of fit. LRT is quite widely used in model selection, for example:

Baayen, R. H. (2008). Analyzing Linguistic Data. A Practical Introduction to Statistics Using R. Cambridge University Press.

Bates, D., Kliegl, R., Vasishth, S., & Baayen, H. (2015). Parsimonious mixed models. arXiv preprint arXiv,1506.04967.

You can also consider using the information criteria. In general, AIC tends to favour more complex models, whereas BIC penalises heavier for model complexity, so typically favours simpler models.

If you are conducting a confirmatory study, and all of your variables are hypothesised to be significant, then I would favour the more complex model, as finding that a variable is not significant is a finding in itself.

Hope this helps.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.