32
$\begingroup$

I'm wondering how to interpret the coefficient standard errors of a regression when using the display function in R.

For example in the following output:

lm(formula = y ~ x1 + x2, data = sub.pyth)
        coef.est coef.se
(Intercept) 1.32     0.39   
x1          0.51     0.05   
x2          0.81     0.02   

n = 40, k = 3
residual sd = 0.90, R-Squared = 0.97

Does a higher standard error imply greater significance?

Also for the residual standard deviation, a higher value means greater spread, but the R squared shows a very close fit, isn't this a contradiction?

$\endgroup$

1 Answer 1

60
$\begingroup$

Parameter estimates, like a sample mean or an OLS regression coefficient, are sample statistics that we use to draw inferences about the corresponding population parameters. The population parameters are what we really care about, but because we don't have access to the whole population (usually assumed to be infinite), we must use this approach instead. However, there are certain uncomfortable facts that come with this approach. For example, if we took another sample, and calculated the statistic to estimate the parameter again, we would almost certainly find that it differs. Moreover, neither estimate is likely to quite match the true parameter value that we want to know. In fact, if we did this over and over, continuing to sample and estimate forever, we would find that the relative frequency of the different estimate values followed a probability distribution. The central limit theorem suggests that this distribution is likely to be normal. We need a way to quantify the amount of uncertainty in that distribution. That's what the standard error does for you.

In your example, you want to know the slope of the linear relationship between x1 and y in the population, but you only have access to your sample. In your sample, that slope is .51, but without knowing how much variability there is in it's corresponding sampling distribution, it's difficult to know what to make of that number. The standard error, .05 in this case, is the standard deviation of that sampling distribution. To calculate significance, you divide the estimate by the SE and look up the quotient on a t table. Thus, larger SEs mean lower significance.

The residual standard deviation has nothing to do with the sampling distributions of your slopes. It is just the standard deviation of your sample conditional on your model. There is no contradiction, nor could there be. As for how you have a larger SD with a high R^2 and only 40 data points, I would guess you have the opposite of range restriction--your x values are spread very widely.

$\endgroup$
2
  • $\begingroup$ Excellent and very clear answer! So basically for the second question the SD indicates horizontal dispersion and the R^2 indicates the overall fit or vertical dispersion? $\endgroup$
    – upabove
    Commented Nov 11, 2011 at 8:42
  • 7
    $\begingroup$ @Dbr, glad to help. Usually we think of the response variable as being on the vertical axis and the predictor variable on the horizontal axis. With this setup, everything is vertical--regression is minimizing the vertical distances between the predictions and the response variable (SSE). Likewise, the residual SD is a measure of vertical dispersion after having accounted for the predicted values. Finally, R^2 is the ratio of the vertical dispersion of your predictions to the total vertical dispersion of your raw data. $\endgroup$ Commented Nov 11, 2011 at 16:14

Not the answer you're looking for? Browse other questions tagged or ask your own question.