3
$\begingroup$

Can we say

$$\text{Var}(\beta_1) = \text{Var}\left(\frac{\sum (x_i-\bar x)y_i}{\sum (x_i- \bar x)^2}\right) = \left(\frac{\sum (x_i-\bar x)}{\sum (x_i- \bar x)^2}\right)^2 \text{Var}(y_i) \;\;??$$

I am not sure if I can separate the $x$'s i from $\sum (x_i-\bar x)y_i$. This expression seems like a linear combination of $y_i$'s. Is this legit because every $y_i$ follows the same distribution?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ I've used LATEX to give your question mathematical formatting. Please double-check if it stills says what you intended. $\endgroup$
    – Sycorax ♦
    Nov 2, 2014 at 19:18
  • $\begingroup$ Thanks I was editing at the same time. Did I overwrite yours? $\endgroup$
    – user2502240
    Nov 2, 2014 at 19:19
  • 1
    $\begingroup$ It appears that you have overwritten me. This is fine. The most important part is two fold (1) is the question legible to people who might be able to answer and (2) does the formatted text say what you want? $\endgroup$
    – Sycorax ♦
    Nov 2, 2014 at 19:21
  • 1
    $\begingroup$ Don't conflate the parameter with its estimate; you'll get very confused. $\endgroup$
    – Glen_b
    Nov 2, 2014 at 21:46
  • $\begingroup$ Briefly: not quite. It's not quite an identical Q but answers here may be of some assistance. $\endgroup$
    – Glen_b
    Nov 2, 2014 at 21:53

2 Answers 2

Reset to default
3
$\begingroup$

This appears to be simple linear regression. If the $x_i$'s are treated as deterministic, then things like "variance" are not associated with them, and so the expression holds, under the additional assumption that the the error term (and hence $y$ also) has identical distribution for all $i$, and also, that the error terms (and hence $y$ also) are independent for all $j\neq i$.

For compactness, denote $$z_i = \frac{x_i-\bar x}{\sum (x_i- \bar x)^2}$$

Then

$$\text{Var}(\beta_1) = \text{Var}\left(\sum z_iy_i\right)$$

The assumption of deterministic $x$'s permits us to treat them as constants. The assumption of independence permits us to set the covariances between $y_i$ and $y_j$ equal to zero. These two give

$$\text{Var}(\beta_1) = \sum z_i^2\text{Var}(y_i)$$

Finally, the assumption of identically distributed $y$'s implies that $\text{Var}(y_i)= \text{Var}(y_j) \;\; \forall i,j$ and so permits us to write

$$\text{Var}(\beta_1) = \text{Var}(y_i)\sum z_i^2$$

Improve this answer
$\endgroup$
3
$\begingroup$

There are a few way to approach this. Alecos' method is simple and clear. Here's another:

$\hat{\beta}_1=\frac{\sum (x_i-\bar x)y_i}{\sum (x_i- \bar x)^2}=\frac{\sum (x_i-\bar x)y_i}{S_{xx}}=\frac{\sum d_iy_i}{S_{xx}}$

$\text{Var}(\hat{\beta}_1) = \text{Var}\left(\frac{\sum (x_i-\bar x)y_i}{\sum (x_i- \bar x)^2}\right)=\frac{1}{S_{xx}^2}\text{Var}\left(\sum d_iy_i\right)=\frac{1}{S_{xx}^2}(\sum d_i^2)\text{Var}\left(y_i\right)$

You can then inspect $\sum d_i^2$ in order to do some cancelling out in the formula, and you can substitute a value for the variance of the observations.

Compare with the sample estimate of the standard error, $s_{\hat\beta}$ here, which should be similar up to the two obvious differences.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.