# Big Misconception in the use of REML to estimate Variance Components?

I often hear from my classmates, and even in resources on the internet such as in the abstract to this paper, what I now believe to be a misconception regarding the motivation of using REML (Restricted Maximum Likelihood) to estimate variance components in a random effects model. I want to check with the community that my understanding is correct.

People often start by saying that the "ANOVA Method" (when we use Expected Mean Squares to construct the point estimate of the variance component) can produce negative estimates, which is true. But then they proceed to say that the purpose of REML is to avoid this. I don't think this is strictly correct.

For example, in the model $Y_{ij} = \mu_i + \epsilon_{ij}, \mu_i \sim^{iid} N(\mu,\sigma^2 _\mu ), \epsilon_{ij} \sim^{iid}N(0,\sigma^2),$ with $\mu_i$ and $\epsilon_{ij}$ independent, the REML estimate can be shown to be:

$$\hat{\sigma}^2_\mu =\frac{1}{n}\big[ MS_{Trt} - MSE \big]$$

which can clearly be negative when $MSE > MS_{Trt}$.

In general, the maximum of the restricted likelihood can occur at a point when the estimate of the variance component is negative.

Instead, the purpose of REML is to achieve a less-biased estimate of the variance component as compared to ML. And further, likelihood methods are used in general over the ANOVA method since the ANOVA method only works well in the balanced case; in the unbalanced case, the ANOVA decomposition into sums of squares is not unique so neither is the estimate. (Most of the discussion in this paragraph can be found in Faraway's "Extending the Linear Model with R").

• What is your question? – Tim Jul 20 '16 at 19:41
• Thanks for asking. My question is, simply put, "is everything I wrote above correct? And is what I call a 'misconception' incorrect?" Since I have professors, colleagues, and online resources suggesting the misconception I state above, I am seeking confirmation about what is correct. Does that make sense? – RMurphy Jul 20 '16 at 20:02
• REML can and will produce negative estimates for variance components if the likelihood surface says to go in that direction. See Searl, Casella & McCulloch. Only the sum is implicitly required to be positive. – Nicholas Mancuso Sep 27 '17 at 21:42