I'm using lrtest to compare two models in Stata. When I get a $p$-value of less than 0.05, does it mean one model is better than the other one? But which one is better? The one that is nested in the other one?
$\begingroup$
$\endgroup$
3
-
2$\begingroup$ I'm voting to close this question as off-topic because it is too elementary and only requires a short answer. $\endgroup$– Michael R. ChernickCommented Jan 5, 2017 at 4:58
-
3$\begingroup$ You could've added your short answer as well!!!! $\endgroup$– lsadelaideCommented Jan 5, 2017 at 6:38
-
1$\begingroup$ The question may also be off topic because it may be primarily a programming issue. I am not familiar with Irteat and that is why I hesitate to answer. The short answer is that in statistics we form a null and alternative hypothesis. The way Jerzy Neyman set this up is to try to find evidence to reject the null hypothesis. A high p-value means you can't reject and a low one means you can reject. We don't view this in terms of which hypothesis is better. Let's just say that if you reject the null hypothesis you favor the alternative. So it is just a matter of identifying the null. $\endgroup$– Michael R. ChernickCommented Jan 5, 2017 at 6:57
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
6
If (and only if) this pertains to a Likelihood Ratio test between two models (fitted by likelihood maximization techniques), a significant test would mean the 'alternative' model has a better fit (read: higher likelihood) on your data than the 'null hypothesis' model (see Michael Chernick's comment).
Please note that this only applies when comparing nested models (e.g. $a+\beta_1x_1$ vs $a+\beta_1x_1+\beta_2x_2$), and that looking at the likelihood does not take into account how much data / parameters one has fitted. The latter is important when you want to ensure parsimony (i.e. picking the simplest model possible without losing a 'significant' amount of fit on your data). If you want to take into account parsimony, take a look at Akaike's Information Criterion.
-
$\begingroup$ Just added some formatting. Feel free to remove if it changes the intent of your post. $\endgroup$– AsheCommented Jan 5, 2017 at 13:54
-
$\begingroup$ Starts to go wrong in the 2nd paragraph. Consider the relation between AIC and LRT for nested models: e.g. with a difference of 1 d.f. between models, model selection using AIC is exactly equivalent to using an LRT with significance level of about 15.73%. How then is it only AIC that's able to ensure parsimony? $\endgroup$– Scortchi ♦Commented Jan 6, 2017 at 23:46
-
$\begingroup$ @Scortchi you are absolutely right. I wanted to say that looking at likelihood alone (which is not the same as the LRT) will not be sufficient. I've edited the answer. And yes, for comparing nested models the LRT does take into account the difference in dfs used. However what the OP described is not a nested situation. So I feel the AIC is the way to go. Consequently if you use the AIC, then no testing of a significant difference of the AIC is possible. Or do you know some way? $\endgroup$– IWSCommented Jan 7, 2017 at 16:52
-
1$\begingroup$ ps. Seems I've confused two questions: OP question did not concern non-nested models. So, in this case, both the LRT and the AIC are possible ways of modelbuilding/predictorselection $\endgroup$– IWSCommented Jan 9, 2017 at 8:59
-
1$\begingroup$ See Comparison of log-likelihood of two non-nested models. $\endgroup$– Scortchi ♦Commented Jan 9, 2017 at 13:22