Comparisions between REML and ML

Question

I have two questions and really appreciate your answers.

If Maximum Likelihood (ML) and Restricted Maximum Likelihood (REML) methods have the same fixed effects in a model, are the results from these two approaches comparable?

When ML and REML are applied with the same fixed effects, do they yield the same variances for the fixed-effect estimates?

Thank you!

Answer 1

More often than not, the point estimates for the non-varying coefficients will be similar in the two approaches, if not exactly the same. However, standard errors and random effect variance estimates will show differences, especially for smaller sample sizes.

When the number of groups is 50 and above, you should see very little difference in the coefficients and standard errors for all parameters, both fixed and random. But the more the group sample size goes below 50, differences will be apparent.

Another contributing factor to the coefficients is whether you have balanced or unbalanced data. In blanced data, you have the same number of observations within each unit whereas in unbalanced data, the number of observations varies by unit.

An excellent and very approachable paper on these issues is McNeish (2017).

As a little data example of how these factors play a role, I played with the pig weight data that comes with lme4 and is available in Stata. Below is a table of estimates from the following three models, estimated by REML and ML:

1. Full sample - 48 pigs, each contributing weight data for 9 weeks
2. Reduced sample - randomly picking 40% of the pigs in the data (N = 19), each with 9 weeks of data.
3. Unbalanced reduced sample - randomly recoding 15% of the weight data points to missing.

                        1. (REML)      1. (ML)     2. (REML)    2. (ML)     3. (REML)     3. (ML) 
                          b/se         b/se         b/se         b/se          b/se         b/se
--------------------------------------------------------------------------------------------------
week                     6.209896     6.209896     6.194737     6.194737     6.188985     6.189036
                       (.0920383)   (.0910745)   (.1721518)   (.1675603)   (.1752122)   (.1705565)
_cons                    19.35561     19.35561     19.14912     19.14912     19.12613      19.1259
                       (.4038678)   (.3996387)   (.5166584)   (.5028784)   (.5263635)   (.5124757)
--------------------------------------------------------------------------------------------------
sd(week)                 .6164383     .6095286     .7325558     .7120406     .7399749     .7190291
                       (.0680542)   (.0666874)   (.1281298)   (.1215559)    (.130784)   (.1241125)
sd(_cons)                2.643194     2.612157     2.057711     1.991792     2.051406     1.983608
                       (.3057587)   (.2997895)   (.4115527)   (.3921792)   (.4259403)   (.4065086)
corr(week,_cons)        -.0634379    -.0618257    -.4270767    -.4251242    -.4343511    -.4319344
                       (.1588762)   (.1575911)   (.2126164)   (.2086346)   (.2152318)   (.2116003)
sd(Residual)             1.263657     1.263657     1.259782     1.259782     1.316938     1.316837
                       (.0487466)   (.0487466)   (.0772422)   (.0772422)   (.0898902)   (.0898709)

--------------------------------------------------------------------------------------------------
N                             432          432          171          171          145          145
--------------------------------------------------------------------------------------------------

For those interested, the Stata code to reproduce these results is below:

version 18.0 
webuse pig, clear

set seed 32604

** Full sample (N = 48)
mixed weight week || id: week, cov(un) reml
eststo full_reml

mixed weight week || id: week, cov(un) 
eststo full_ml

** Reduced L2 sample (40% of original - N = 19)
gsample 40, percent wor cluster(id) gen(in_red) 

egen pick1id = tag(id)
tab in_red if pick1id==1

mixed weight week if in_red==1 || id: week, cov(un) reml stddev
eststo red_reml

mixed weight week if in_red==1 || id: week, cov(un) stddev
eststo red_ml

** Unbalancing the data 
gsample 15 if in_red==1, percent wor gen(to_missing)

replace weight = . if to_missing==1
sum weight if in_red==1 & weight !=.


mixed weight week if in_red==1 || id: week, cov(un) reml stddev
eststo uneqred_reml

mixed weight week if in_red==1 || id: week, cov(un) stddev
eststo uneqred_ml