Robust standard errors in multiple regression

Question

I use Andrew F. Hayes' macro for SPSS (HCREG at http://www.afhayes.com/spss-sas-and-mplus-macros-and-code.html) to perform multiple regression analyses with robust standard errors.

The information I get in the output is limited compared to the one SPSS provides when applying multiple regression. For example, there are no standardized coefficients (Beta) for predictors and there is no adjusted R-squared.

Can I assume that the standardized coefficients will be the same as in the model without robust standard errors? If not, does anyone know how can I go about computing them by hand?

I have the same question for the adjusted R-squared.

Any help would be greatly appreciated! Thank you very much in advance.

Answer 1

The adjustments are only to the standard errors of the regression coefficients, not to the point estimates of the coefficients themselves. So you can gather the requested statistics from the traditional OLS output in SPSS. The Hayes and Cai, 2007 paper elaborates on this, as well.

To note, perhaps it is a difference between fields but I almost always see these types of standard errors referred to by their originators (Huber, White and Eicker). There are other types of "robust" estimates and standard errors though (e.g. estimated by the jack-knife or bootstrapping). Sometimes these other estimators do have different point estimates for the coefficients and the standard errors of the coefficients (not always though).

Answer 2

The central issue is that these "robust" standard errors are derived to deal with heteroskedasticity. If your $\epsilon_i$ in regression do not have a common variance, there is no point in defining an estimate of $\sigma^2$, as there is no population analogue for it, and the sum of squared residuals, central to definition of both $R^2$ and the standardized coefficients, does not make much sense, either. For sure, there is no longer an $F$-distribution to be used in tests on reduction on the sum of squares. My guess is that SPSS duplicates Stata's behavior on this; Stata has had it for what, 20 years? This FAQ is written by the author of Stata's robust standard errors in 1998 when they had it up and running for a couple of releases; this and some other FAQs concerning robust standard errors are worth looking at.

Disclaimer: I don't like the term "robust standard errors" very much. If at all. I positively hate it. There is no single unique definition of "robust" standard errors; rather, there is a whole class of these standard errors, and you can always find something wrong with any member of the class. White/Eicker standard errors are consistent under heteroskedasticity, but are biased in small samples even under homoskedasticity, and inconsistent under serial correlation or clustering. Newey-West standard errors correct for autocorrelation, but only up to a certain lag. Any of these standard errors are not robust in the Swiss meaning of the word: outliers going to infinity still screw them up. I think economists tend to use more cautious ``heteroskedasticity-consistent'' terminology.