Suppose we have a correlated outcome $\mathbf{y}$ and a bunch of predictors $\mathbf{X}$. For some reason, we know the variance/covariance matrix of the error term $(\epsilon)$, say $\mathbf{V}$.

In this scenario, it is reasonable to utilize the Generalized Least Squares.

Through Cholesky decomposition, we can calculate $\mathbf{P'}\mathbf{P}=\mathbf{V}$. It follows that we can use Ordinary Least Squares for the model $\mathbf{z}=\mathbf{Q\beta}+\mathbf{f}$, where $\mathbf{z}=\mathbf{P^{-1}y}$, $\mathbf{Q}=\mathbf{P^{-1}X}$ and $\mathbf{f}=\mathbf{P^{-1}\epsilon}$.

What would be in this case, if any, the advantage(s) introduced by using a robust variance estimator for the model $\mathbf{z}=\mathbf{Q\beta}+\mathbf{f}$? Would it relax the assumption that the matrix $\mathbf{V}$ we used is actually the right variance/covariance matrix for $\epsilon$? Does it even make sense using a robust variance estimator in this scenario, since we (pretend to) know $\mathbf{V}$?


1 Answer 1


Pretending to know the true variance is always a funny thing to see. I would never trust that sort of an assumption, unless it comes from a carefully designed experiment where an artificial correlation structure was imposed (although frankly I cannot really imagine a practical set up that would lead to this).

From the GEE perspective, you should be able to get more efficient estimates when your assumption about $\bf V$ is correct, as compared to OLS, but you would still want to use the sandwich variance estimator in the (highly unlikely) case that you are mistaken about the covariances. Usually, robustness to model assumptions is considered a greater issue than efficiency, unless you have really tiny sample sizes, and any 20% efficiency gain is a big deal. So I would run this with as the GLS (or feasible GLS, if you only know the structure of $\bf V$, but not the specific parameter values), but still correct for clustering using the sandwich variance estimator.

  • $\begingroup$ Well, you're right, I'm only pretending I know the matrix V. What I'm doing is a meta-analysis. For each study I estimate a linear trend for the association between an exposure (reported in categories, the X) and the log-relative risks (the y). Since the (log-)relative risks have a reference category in common, they are correlated. The correlation matrix V is estimated using Greenland's method (aje.oxfordjournals.org/content/135/11/1301.abstract) and passed to a GLS regression, where V is supposed to be known. This is why I was thinking of using the robust variance estimator. $\endgroup$
    – boscovich
    Sep 27, 2012 at 19:25
  • $\begingroup$ @StasK When I saw this question I thought this sounds like a great question for Macro. But I guess he is retired for a while. These are especially the time when we would miss him. $\endgroup$ Sep 27, 2012 at 20:34
  • $\begingroup$ @andrea, I see. I know little of meta-analysis, so I can't say whether these assumptions are making enough sense. It will probably be little harm to try OLS and GLS with this unusual assumed structure, and see if the results from one or the other are drastically different. They shouldn't be -- the correlations that GLS exploits would have to be like 0.8 to give the GLS a strong edge, but who knows. $\endgroup$
    – StasK
    Sep 28, 2012 at 14:42
  • $\begingroup$ @StasK thank you very much, I'll bother you with one last question. Am I correct if I say, then, that I don't have only 2 approaches to deal with my problem (OLS+robust standard error or GLS), but rather I can also use GLS+robust SE? Do you have any reference about GLS+robust SE? It seems to me that this third option (GLS+robust SE) is never mentioned in papers and only the other 2 are considered (OLS+robust SE and GLS). $\endgroup$
    – boscovich
    Sep 29, 2012 at 12:20
  • $\begingroup$ One last thing: unfortunately, the sample sizes I'm working with are usually ridiculously small: 4-7 points, as those are the number of dose-specific relative risks typically reported in epidemiological papers (and I have to fit a trend passing through those data). Thus, I am not only worried about the SE, but also about the $\beta{}$ coefficient for the trend. This is why I would exclude OLS+robust SE as a possible solution to my problem. $\endgroup$
    – boscovich
    Sep 29, 2012 at 12:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.