0
$\begingroup$

I have a question about what defines “nested models” that can be compared using the likelihood ratio test (LRT). For example, given the 3 linear models…

M1: Y = Intercept + B1 * X1 + B2 * X2
M2: Y = Intercept + B1 * X1
M3: Y = Intercept + B3 * (0.5 * X1 + 0.5 * X2)

M1 and M2 are nested models because M2 is a special case of M1 with B2 = 0. M1 and M2 can be compared using the LRT with 1 degree of freedom.

The model M3 might arise if you decided to model Y as a function of a pre-specified linear combination of X1 and X2 (in this example I’ve used the mean of X1 and X2). Are M1 and M3 nested models? My thinking is that M3 is a special case of M1 with B1 = B3 * 0.5 and B2 = B3 * 0.5. Can M1 and M3 be compared using the LRT? With 1 degree of freedom?

$\endgroup$
1
$\begingroup$

Yes, M1 is nested in M3.

The models satisfying M3 are the precisely the subspace of models satisfying M1 that also satisfy B1=B2. That's a one-dimensional restriction, so you can test the null hypothesis B1=B2 by comparing M1 and M3 with 1 df.

$\endgroup$
0
$\begingroup$

By specifying the linear combination, you break the nesting.

Say that $M1$ turns out to be $\hat{y}=3-x_1+2x_2$.

$M3$ can’t ever be that equation, as $M3$ requires the coefficients on $x_1$ and $x_2$ to be equal.

You can adjust the linear combination of $x_1$ and $x_2$ but will always be able to find fitted values in $M1$ that cannot happen in $M3$.

$\endgroup$
2
  • $\begingroup$ Thanks for your answer. My understanding is that nested models are models that can be obtained by restricting a parameter in a more complex model to be 0. Suppose we rewrite your M1 example as Y = 3 + (0.5 * X1 + 0.5 * X2) - 1.5 * X1 + 1.5 * X2. From this perspective, is my M3 example Y = 3 + (0.5 * X1 + 0.5 * X2) nested? (because the second X1 coefficient (-1.5) and the second X2 coefficient (1.5) are 0). My comment looks kinda wrong to me, but I'm not sure why.. $\endgroup$
    – Scarper
    Mar 25 '20 at 15:39
  • $\begingroup$ I suppose so, but then you have perfect collinearity by using the $x_1$ variable twice and putting different parameters on each instance. I don’t know for sure what perfect collinearity does to a likelihood ratio test, but I suspect that it isn’t good. $\endgroup$
    – Dave
    Mar 25 '20 at 16:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.