3
$\begingroup$

The adjusted $R^2$ is not shown when a regression with robust standard errors is calculated in Stata. This is surprising to me since the value of the $R^2$ is unaffected in regressions with robust standard errors.

Is there any statistical reason for not quoting the adjusted $R^2$ when using robust standard errors in regression?

Furthemore if I add more variables the F test disappears.(e.g. with 9 variables it shows but not with 13), Is this for the same reason? How can we report such results and deal with this issue?

$\endgroup$
11
$\begingroup$

If you specify the robust option, you are telling Stata that you don't really believe the errors are homoskedastic. There are several implications:

  • The sums of squared residuals are still used to drive the estimation (you minimize them).

  • As every observation has its own variance, the sum of the squared errors is no longer distributed as a $\chi^2$, and the ratios of the model and residual sums of squares are no longer distributed as an $F$.

  • Since different observations contribute different amount of information, there isn't any way now to correct $R^2$ for the prognostic value the way $R^2_{\rm adj}$ does: one observation $\neq$ one degree of freedom.

Essentially, as the notion of variance is not quite applicable to the dependent variable (do you want to talk about the conditional variance of $Y$, i.e., the variance of the error terms, which, as we said, is not constant any more; or do you want to talk about the total variance, which, inconveniently, depends on the distribution of the explanatory variables), the whole concept of $R^2$ starts breaking down in its meaning.

$\endgroup$
  • 1
    $\begingroup$ Plus adjusted R2 is a horrible thing. $\endgroup$ – Jeremy Miles Aug 26 '13 at 18:10
  • $\begingroup$ This is a normative judgement, @JeremyMiles. Obviously, a lot of people expect to see it... and also want to see it in Poisson and logistic regression, as well. $\endgroup$ – StasK Aug 26 '13 at 20:54
  • $\begingroup$ @StasK On the same logic, the $R^2$ is shown but. Its only the adjusted $R^2$ which is not shown. And so is the F-test. So technically, when I report the data in this case, show I report the $R^2$ (ignoring the adjusted $R^2$?). How am I to report the data given I'm using the robust se. $\endgroup$ – Cesare Camestre Aug 26 '13 at 21:18
  • $\begingroup$ @JeremyMiles Explain why you think so. $\endgroup$ – Cesare Camestre Aug 26 '13 at 21:18
  • 1
    $\begingroup$ According to this post which I have just found, "the F statistic, for instance, is no longer based on sums of squares; it becomes a Wald test based on the robustly estimated variance matrix" (is this correct?). What is puzzling is that the $R^2$ is still shown. $\endgroup$ – Cesare Camestre Aug 26 '13 at 21:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.