2
$\begingroup$

I have been reading Mixed Effects Models and Extensions in Ecology with R by Zuur et al. where departures from iid errors (heteroscedacity and/or correlation) in linear regression (and glm) are dealt with using gls() in the nlme package and specifically the process advocated is to set up an error correlation structure and/or weights for the model explicitly based on 1) findings from examining a residual plot or ACF, 2) conducting a likelihood ratio test using nested models or 3) AIC for non-nested models. In this manner, the analyst is able to iterate on the model selection process until the (normalized) residuals appear to satisfy the assumptions of the model.

This approach contrasts with using a robust standard error such as this blog post http://rforpublichealth.blogspot.com/2014/10/easy-clustered-standard-errors-in-r.html

My question is if both approaches are valid and simply represent different approaches or if one of the approaches is considered superior?

$\endgroup$
  • 5
    $\begingroup$ Without knowing the specific application, use of GLS and use of robust covariance matrix have one significant difference: GLS estimates the covariance structure in stage 1 and uses it to change the coefficient estimates in stage 2; meanwhile, use of robust covariance matrix leaves the coefficient estimates intact but expands confidence intervals to account for the violated OLS assumptions. Thus GLS goes one step further and attempts to fix the coefficient estimates while use of robust covariance matrix stops at acknowledging there is a problem and accordingly expanding confidence intervals. $\endgroup$ – Richard Hardy Oct 27 '15 at 20:50
  • $\begingroup$ I dont have a specific application it is more a general tool question I guess as in what to do with heteroscedacity and/or correlation in the errors...GLS superior in this regard then? $\endgroup$ – B_Miner Oct 27 '15 at 21:04
  • $\begingroup$ As long as you are able to get a reasonable estimate of the covariance structure, GLS should be superior. However, in small samples GLS is sometimes found to be inferior. So I guess it depends. $\endgroup$ – Richard Hardy Oct 27 '15 at 21:15
  • $\begingroup$ OK, so no-one would say "you used GLS, how 2005 of you?" ;-) Make this an answer and I will accept. $\endgroup$ – B_Miner Oct 27 '15 at 22:36
5
$\begingroup$

Without knowing the specific application, use of GLS and use of robust covariance matrix have one significant difference:

  • GLS estimates the covariance structure in stage 1 and uses it to change the coefficient estimates in stage 2;
  • use of robust covariance matrix leaves the coefficient estimates intact but expands confidence intervals to account for the violated assumption of i.i.d. errors.

Thus GLS goes one step further and attempts to fix the coefficient estimates while use of robust covariance matrix stops at acknowledging there is a problem and accordingly expanding confidence intervals.

As long as you are able to get a reasonable estimate of the error covariance structure, GLS should be superior. However, in small samples GLS is sometimes found to be inferior. So I guess it depends.

$\endgroup$
  • $\begingroup$ Could it be that GLS allows for heteroscedasticity and correlation, but still assumes ( multuvariate) normality and thus 'thin' tails (few outliers) while robust regression is more suited for errors with distributions with fatter tails? $\endgroup$ – user83346 Oct 28 '15 at 6:16
  • $\begingroup$ @fcop, Well, asymptotically the error distribution does not matter for GLS (similarly to OLS) because of the central limit theorem. However, I do not really know how robust regression takes care of fat tails; does it at all? If you know that, please share (comment or add another answer). $\endgroup$ – Richard Hardy Oct 28 '15 at 6:35
  • $\begingroup$ see e.g. section 'parametric alternatives' on en.wikipedia.org/wiki/Robust_regression $\endgroup$ – user83346 Oct 28 '15 at 8:35
  • $\begingroup$ @fcop, How does that work in practice? Having assumed a particular error distribution (without specifying the exact parameters), is maximum likelihood used then to obtain the estimates of regression parameters $\hat\beta$ (together with estimates of the distribution parameters, e.g. the d.f. of a $t$-distribution)? $\endgroup$ – Richard Hardy Oct 28 '15 at 8:53
  • 2
    $\begingroup$ I think this is quite wrong and contradicts the basic justification and reasoning behind GLS. Suppose the errors are correlated and we are able to capture that reasonably well. OLS ignores this and is inefficient. GLS takes explicit account of this and is efficient. Both are consistent. The fact GLS has different point estimates from OLS reflects the failure of OLS, not of GLS, because OLS incorrectly assumes no correlation -- while GLS correctly adjusts for the present of correlation. $\endgroup$ – Richard Hardy Oct 28 '15 at 14:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.