Encompassing Tests to compare models in R I recently read a paper in which an academic compared two competing models of voting theory. One of the advanced diagnostic tests he did was to try and see whether either of the models could "formally encompass one another". The way he was able to interpret the results was by comparing the reduction in residual deviance or AIC when M1 was added to M2, as opposed to M2 being added to M1.
I have joylessly been trying to do this in R. Does anybody have any experience with encompassing tests? Effectively all I am attempting to do is compare two models which are non-nested. The models are completely different apart from the control variables, therefore anova(m1, M2) does not work.
Can anyone advise or suggest ways to proceed?
 A: If you have two competing linear models with different regressors, say:
m1 <- lm(y ~ x1 + x2, data = ...)
m2 <- lm(y ~ z1 + z2, data = ...)

then the encompassing model is
m12 <- lm(y ~ x1 + x2 + z1 + z2, data = ...)

and the encompassing test compares
anova(m1, m12)
anova(m2, m12)

The "hope" would be that one of the models (m1, m2) is significantly worse than the encompassing model m12 while the other is not. The one that is not significantly worse would be preferable then. However, it may happen that both models are significantly worse than m12 in which case neither model alone is entirely convincing.
If you want a convenience function to carry out this test in R, you can use the encomptest() function from the lmtest package:
encomptest(m1, m2)

The package also provides other tests for non-nested model comparisons, e.g., jtest() and coxtest().
A: There are different encompassing tests. I'll give you an example of forecast encompassing test from "A Companion to Economic Forecasting" edited by Michael P. Clements, David F. Hendry, see Eq (14.2) on p.302. 
Let's say you have competing forecast $\hat y_{1,t+1}$ and $\hat y_{2,t+1}$, and the actual observations $y_{t+1}$. Run a regression: 
$$y_{t+1}=\alpha_0+\alpha_1 \hat y_{1,t+1}+\alpha_2 \hat y_{2,t+1}+u_{t+1}$$
Under $H_0$: model 1 encompasses model 2, you have $\alpha_1=1$ and $\alpha_2=0$. This is a simple linear restriction test, that can easily be done in R or any other stat package.
Note, that you don't need to know anything about the models, their independent variables etc. All you need is their forecasts and actual observations.
