Using Weighted Least Squares with Robust Standard Errors

Question

Weighted least squares (WLS) and robust standard errors are sometimes presented as alternative approaches for obtaining reliable standard errors of estimates of regression coefficients in the presence of heteroscedasticity. However, I notice that my software (gretl) offers robust standard errors as an option when using WLS.

A situation in which it seems this might be useful is where, in a regression of Y on X, there is a clear reason for heteroscedasticity, for example a scale effect such that larger values of Y are expected to be associated with larger variances. One might then use WLS, giving a higher weighting to observations with smaller Y (or, perhaps better, to observations with smaller E[Y | X], as inferred from an initial OLS regression). However, it might be found that the WLS residuals suggested some remaining heteroscedasticity that the weighting had not eliminated. This would suggest that the standard errors estimated by WLS might not be entirely reliable, and to address this one might opt for robust standard errors (rather than attempting to do so via some more complex weighting pattern).

Question: Assuming the number of observations is reasonably large (over 100, say), are there any pitfalls in using WLS with robust standard errors when estimating standard errors of regression coefficients?

Answer 1

I break your concerns about the estimator into two areas: efficiency and asymptotic validity. I'll define a procedure as asymptotically valid if the point estimates are consistent, and the estimated variance-covariance is consistent. An extension of Alecos's arguments show, the robust (ie, sandwich) standard errors result in asymptotic validity, regardless of the assumed weighting matrix, and in fact this result even holds for clustered/correlated data (as long as independence holds on at the uppermost level of clustering).

I'll define the efficiency of the estimate as the true asymptotic variance/covariance matrix of the coefficients. Of course, from Gauss-Markov we know that only when you select weights proportional to the inverse conditional variance of each observation will you achieve the best limiting unbiased limit.$^1$ So based on first order, asymptotic concerns, we may just take the best stab at estimating the weightings we can, then go ahead and just robust standard errors to guard against mistakes in the weights.

To say anything more refined then this we need to think of second-order asymptotic or finite sample concerns. An example of a second order concern might be "variance of the variance." While I don't have the inclination to try to make Aleco's argument rigorous, I believe it does hold--that when you estimate additional, unnecessary parameters you will introduce additional variance in the remaining parameters. (You might be able to make it rigorous by considering schur decompositions of blocks of the information matrix?) So there is probably a second-order bias-variance tradeoff present: when you use the robust standard errors, you eliminate bias in the standard errors, at the cost of maybe more variance in them.

Most people seem to care more about the bias than the variance, but if this tradeoff is important, then the only advice I have to offer is to simulate or bootstrap see how much it might matter in your application. There's probably some additional theory extant or to be developed that could offer some advice by using higher-order asymptotics, but that's beyond my paygrade.

$^1$ Proof here, apparently originally due to Aitchen.