Linear regression of 1 independent variable produces the best fit line given the samples available. In Excel, the automatic linear regression of 1 independent variable yields some values such as these below. (Sorry that the image is not cropped.) Given that there is only one such line possible for any optimizing algorithm, there is only one optimal pair that is the slope and the y-axis-intercept. There is only one optimal a+bx, there is only one optimal a, there is only one optimal b, so what is the meaning of the standard error (of a)?
$\begingroup$
$\endgroup$
0
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
You have just a single variable in this linear regression:"excesslnst". It has a regression coefficient of 0.51; a standard error of 0.026; a t stat of 19; and a P value of 0.000.
All those values are related. And, together they give you information of how statistically significant is the regression coefficient associated with your variable excesslnst.
The standard error of this regression coefficient captures how much uncertainty is associated with this coefficient. Sometimes, outputs also give you a 95% Confidence Interval around that coefficient. In your case, the low frontier of this Confidence Interval would be equal to: 0.51 - 1.96(Standard Error). And, the high frontier of this same CI would be: 0.51 + 1.96(Standard Error). In this case your 95% CI for this regression coefficient would range from 0.46 to 0.56.
The t stat is equal to your regression coefficient divided by its Standard Error. So, 0.51/0.026 = 19. In other words, your regression coefficient stands 19 Standard Errors away from Zero or from being Null. This is a huge statistical distance away from zero. And, a t stat of 19 translates into a very statistically significant regression coefficient with a P value of 0.000.... The latter is calculated using a T distribution function that just needs the Degree of Freedom in your model (number of observations minus number of variables) in addition to the t stat. Excel, R and most other software programs have ready formulas to calculate such P values.
As outlined, the regression coefficient Standard Error, on a stand alone basis is just a measure of uncertainty associated with this regression coefficient. But, it allows you to construct Confidence Intervals around your regression coefficient. And, just as importantly it allows you to evaluate how statistically significant is your independent variable within this model. So, it is really key to allow you to interpret and evaluate your regression model.
You should certainly not confuse the Standard Error of a regression coefficient with the Standard Error of your overall model. The former allows you to build a Confidence Interval around your regression coefficient. The latter allows you to build a Confidence Interval around your regression model estimates.
-
$\begingroup$ "Best fit" suggests there is one solution so why is there uncertainty associated with this regression coefficient? $\endgroup$– H2ONaClCommented Jun 29, 2017 at 1:45
-
$\begingroup$ Instead of Best fit, I would use the more relative expression Goodness-of-fit. Regression does minimize the sum of the square of the residuals. But, this does not mean there is not some remaining uncertainty about the derived regression coefficients. Those regression coefficients and their respective standard errors are all calculated and solved for using a closed form Matrix algebra algorithm. $\endgroup$– SympaCommented Jun 29, 2017 at 4:04