Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

If I have a simple regression model: $$Y = \beta_1 + \beta_2(\text{income}) + \varepsilon$$

I can calculate the $\text{Var}(\hat{\beta_2})$ quite easily with a formula. However, what is the practical meaning of $\text{Var}(\hat{\beta_2})$ ? What does it actually tell us? I know that it tells us the variance of the slope $\hat{\beta_2}$, but what does this mean?

Does it mean that from the various samples, when you collate all of the predictions for $\beta_2$, we get an average level of $\beta_2$ and then the variance of this level of $\beta_2$?

share|improve this question
4  
$\text{Var}(\beta_2)=0$ since $\beta_2$ is a fixed number. – Xi'an Jan 27 at 19:39
5  
I think the OP means to ask what the $Var(\hat\beta_2)$ is. @A.Dooley note that the "hats" are important here! Indeed recognizing this fact may help the OP understand what $Var(\hat\beta_2)$ really means! – StatsStudent Jan 27 at 19:43

$\newcommand{\Var}{{\rm Var}}$Simply put, when you perform regression analysis, your analysis is based off of a sample from which there is uncertainty. You could obtain another sample and perform the same regression analysis, but your results will differ slightly. The values or your $\hat\beta$'s will differ a bit for each sample. So $\Var(\hat\beta_2)$ is a measurement of how much variability our estimates would have if we repeated sampling over and over again and fit multiple regression models, each time obtaining a different estimate of $\hat\beta_2$. The $\Var(\hat\beta_2)$ measures how much variability each of the $\hat\beta_2$'s have around their mean.

share|improve this answer
3  
Is it worth mentioning that the usual formulas actually give $Var(\hat \beta \mid X)$. That is, the $X$ data is assumed fixed, and the resampling of datasets is constrained to new $y$'s for those given $x$'s? – Matthew Drury Jan 27 at 20:54
    
+1 @Matthew Drury. Excellent point. – StatsStudent Jan 27 at 21:07
    
@MatthewDrury, unless this is from experimental economics research, I'll be surprised if $X$ is fixed. This kind of stuff in economics is usually from observational studies, hence, $X$ is random. In other words, you can't re-run the economy at different level of income, all you have is what already happened, and you're lucky if you even recorded it. – Aksakal Jan 27 at 22:24
    
@Aksakal but the model is conditioned on $X$ all the same, no? – ssdecontrol Jan 29 at 12:24
    
@ssdecontrol, not the same, there are subtle and not so subtle differences. – Aksakal Jan 29 at 12:28

The variance of $\hat\beta_2$, i.e. the estimator of the true coefficient, tells you how much uncertainty is about the estimated income elasticity of output.

You have very limited amount of data in these kinds of regressions. Say, it's quarterly GDP for a few years. Your sample size is probably tiny, so when you calculate the income elasticity, you better also look at its variance to understand the precision of your estimate. I bet it's miserable.

I'm pretty sure you meant $\hat\beta_2$, not $\beta_2$ which is a constant in OLS, as @Xi'an pointed out. Now, if we want to be pedantic, then in reality $\beta_2$ doesn't have to be and is not a constant. It certainly depends on other things, but in the framework of this model, it is a constant.

share|improve this answer

Even in the idealized, theoretical population there is a spread of the $y$ measurements, which is assumed to follow a normal distribution around the fitted regression line, expressed as $\varepsilon$ in the equation in your original post. The variance of $\varepsilon$ is constant under the Gauss-Markov assumption (homoskadesticity), and expressed as $Var(\varepsilon) = \sigma^2$.

In the calculation of the slope in a OLS regression we depart from a sample. As such, the calculation of the slope will be different depending on the actual values realized in the sample. This is immediately clear in the fact that they are treated as fixed parameters in the construction of the model matrix, and in the linear algebra behind the estimation of the coefficients - the hat matrix: $(X'X)^{-1}X'y$.

In addition, the variance of the estimated values will trickle down from the "population" to the sample. At the sample level, we can only estimate this variance as $s^2(X'X)^{-1}$ with $s^2$ representing the estimate of the variance of the errors through the residuals: $\hat y_i- (\beta_0 + \beta x_i)$ in the case of a single regressor.

At a graphical level, and with some quick lines of code, we can draw 10,000 samples, each one of 100 observations from a known normal distribution, $N(10,5)$, for example, and estimate the spread of the slope of the regression lines. I omitted the intercept.

The true slope was set up at $2.3$.

Here are the plots, which I think illustrate the answer to your question:

enter image description here

enter image description here

The estimation of the slope, $\hat \beta_2$, in your question, does indeed vary with the variance of the estimated points, and ultimately, also with the variance of the points in the "population" around the royal $\beta_2.$

The code is here

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.