If we're working with a linear mixed model (fixed + random effects, normality assumptions) and are interesting in testing whether or not we can remove a fixed effect from the model, since standard asymptotic theory is complicated, my text suggests using a parametric bootstrap for approximating the distribution of likelihood ratio stats under the null rather than a traditional F-test.
However, I'm uncertain about the null hypothesis in this case. Is the null:
- the larger model with the fixed effect under consideration (and the alternative is the smaller model with the particular fixed effect removed)? This captures the spirit of testing whether we can remove this effect, but the models are not nested in the standard order.
- the smaller model without the fixed effect under consideration (and the alternative is the null of #1 above)
When I run a parametric bootstrap under the nulls as in the above two cases, I get different p-values in the two methods (I'm comparing log likelihood differences of my data to the simulated using $2\times [l(x|H_1) - l(x|H_0)]$ for each bootstrap simulation from $H_0,H_1$). But when I run the Kenwood-Roger adjusted F-test (require(pbkr); KRmodcomp(nullmod,altmod)) I see the test is symmetric in its inputs. What's the interpretation of starting with $H_0$ larger vs $H_0$ smaller, and does one better capture the goal of testing whether we can remove a fixed effect?
More generally, it feels artificial to prefer a more complicated model as the default since the smaller model would have more robust estimates of its parameters, smaller variance in the bias/variance tradeoff, and provide a more succinct summary of the data. However, is there an interpretation or general setting in which it makes sense to have a larger model as the default?
