0
$\begingroup$

In the simple linear regression model, $\hat{\beta}$ is the sum of independent normally distributed random variables.

Is it false because in linear regression there is $\beta$ and not $\hat{\beta}$?

$\endgroup$
  • $\begingroup$ I've edited your post. Please comment if I've changed the meaning. $\endgroup$ – Dave Oct 22 '19 at 19:31
  • 1
    $\begingroup$ The model doesn't say anything about $\hat b.$ When you fit it with Ordinary Least Squares, the formula for $\hat b$ exhibits it as a linear combination of the responses $y.$ When you assume those responses are independent and Normally distributed, then indeed $\hat b$ is a linear combination of independent Normal variables: to see that, please search our site for any posts on OLS regression formulas (there are a few thousand of them to choose from). $\endgroup$ – whuber Oct 22 '19 at 19:41
  • 1
    $\begingroup$ Remember that we estimate $\hat{b}$ via $\hat{b} = (X^TX)^{-1}X^Ty$. This means that $\hat{b}$ is a linear combination of the values of $y$. When we assume that each $y_i$ has a normal distribution, then, yes, $\hat{b}$ is just a linear combination of normal random variables. $\endgroup$ – Dave Oct 22 '19 at 19:46
  • 1
    $\begingroup$ @JaneSecuiu What is "^b" if not $\hat{b}$ as the estimate of $b$ in $y=a + bx$? $\endgroup$ – Dave Oct 23 '19 at 13:50
  • 1
    $\begingroup$ ''y = β x + α + ε can be estimated by y^=a^+b^x. (Since the expected value of ε is 0.) b^ then, is [Σ(xi - x¯)(yi - y¯)]/Σ (xi-x¯)2. You may have seen this expressed as Sxy/Sxx. I would hardly call this the sum of independent normal variables. There is no imposition on x to be normal. As long as it correlates linearly with y, the regression will work.'' someone answered to a similar question this.is it false? $\endgroup$ – Jane Secuiu Oct 23 '19 at 14:36
1
$\begingroup$

It is false, but not for the reason you listed.

$\hat{b}$ is an estimate of β, the unknowable parameter. As such you have:

$y$ = β $x$ + α + ε can be estimated by $\hat{y} = \hat{a} + \hat{b}x$. (Since the expected value of ε is 0.)

$\hat{b}$ then, is [Σ($x_i$ - $\bar{x}$)($y_i$ - $\bar{y}$)$]/$Σ ($x_i$-$\bar{x}$)$^2$. You may have seen this expressed as $S_{xy} / S_{xx} $. I would hardly call this the sum of independent normal variables. There is no imposition on $x$ to be normal. As long as it correlates linearly with $y$, the regression will work.

$\endgroup$
  • $\begingroup$ someone to a similar question said: ''Remember that we estimate b^ via b^=((X^T*X)^−1)(X^T)*y. This means that b^ is a linear combination of the values of y. When we assume that each yi has a normal distribution, then, yes, b^ is just a linear combination of normal random variables.'' is this false? $\endgroup$ – Sarah kenwich Oct 23 '19 at 14:29
  • $\begingroup$ @Sarahkenwich A typical (but not mandatory) assumption of OLS regression is that the response variable is conditionally normal. That is, each $y_i$ has a normal distribution. If you make this assumption, then my comment is correct. $\endgroup$ – Dave Oct 23 '19 at 15:01
  • $\begingroup$ one person says it is false, the other that it is true.i do not understand who is correct $\endgroup$ – Jane Secuiu Oct 23 '19 at 15:09
0
$\begingroup$

$$\hat{b} = (X^TX)^{-1}X^Ty $$

When we assume that each $y_i$ has a normal distribution, then $\hat{b}$ is a linear combination of normal random variables and, therefore, a sum of normal random variables.

It is not mandatory that we assume each $y_i$ to be normal. When we assume exponential (for instance) $y_i$, then $\hat{b}$ is a linear combination of exponential random variables instead of normal random variables.

However, normal $y_i$ is a common assumption. When we make this assumption, then $\hat{b}$ is a sum of normal random variables, since $(X^TX)^{-1}X^T $ is a row vector.

In summary, when we make the (extremely common) assumption of normal $y_i$, then $\hat{b}$ is a sum of normal random variables. If we do not make that assumption, then $\hat{b}$ is not a sum of normal random variables.

Depending on the level where your class is operating, the full-credit answer may be either TRUE or FALSE. I would expect a class dealing with regression on one variable to be making the assumption of normal $y_i$, so the full-credit answer would be TRUE.

And since this probably is a question from a class assignment, please include the self-study tag. CV doesn't usually give full answers to homework questions, though it seemed necessary to do so here to clear up some confusion.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.