Linked Questions
11 questions linked to/from Simple Linear Regression: how does $\Sigma\hat{u_i}^2/\sigma^2$ follow chi squared distribution with df (n-2)?
4
votes
0
answers
527
views
How are $\hat{\beta}$ and $\hat{\sigma}^2$ independent in OLS when assuming that $\epsilon \sim N(0, \sigma^2)$? [duplicate]
In Elements of Statistical Learning, pg 47, at the very bottom, it states that $\hat{\beta}$ and $\hat{\sigma}^2$ are statistically independent.
Is this saying that they are independent when ...
- 344
39
votes
3
answers
26k
views
Why is RSS distributed chi square times n-p?
I would like to understand why, under the OLS model, the RSS (residual sum of squares) is distributed $$\chi^2\cdot (n-p)$$ ($p$ being the number of parameters in the model, $n$ the number of ...
- 20.1k
28
votes
3
answers
38k
views
In simple linear regression, where does the formula for the variance of the residuals come from?
According to a text that I'm using, the formula for the variance of the $i^{th}$ residual is given by:
$\sigma^2\left ( 1-\frac{1}{n}-\frac{(x_{i}-\overline{x})^2}{S_{xx}} \right )$
I find this hard ...
- 445
6
votes
2
answers
13k
views
Why divide RSS by n-2 to get RSE?
$$\text{RSE}=\sqrt{\frac{1}{n-2}\text{RSS}}=\sqrt{\frac{1}{n-2}\sum_{i=1}^n (y_i-\hat{y_i})^2}$$
Context: Simple Linear Regression, an intercept and a slope
I have 2 question regarding this issue.
...
- 635
4
votes
1
answer
4k
views
Why error sum of squares has n-2 df (possibly not duplicate, please read on)? (Regression Question Series - Part 4)
In simple linear regression, the error sum of squares is given by
$$
\text{SSE} = \sum_{i=1}^n(y_i - \hat{y_i})^2 \\
\hat{\sigma}^2 = s^2 = \dfrac{\text{SSE}}{n-2}
$$
where $n-2$ is the degrees of ...
2
votes
2
answers
2k
views
Joint distribution of least square estimates $(\hat\alpha,\hat\beta)$ in a simple linear regression model [duplicate]
Let $Y_1,Y_2,\ldots,Y_n$ be independently distributed random variables such that $Y_i\sim\mathcal N(\alpha+\beta x_i,\sigma^2)$ for all $i=1,\ldots,n$. If $\hat\alpha$ and $\hat\beta$ be the least ...
- 9,200
3
votes
1
answer
2k
views
Deriving SSE of Simple Linear Regression is $\chi^{2}$
As per my notes, the key step in the proof that the sum of squares of residuals in regression is $\chi^{2}_{n-2}$ is the fact that $e_{i} = y_{i} - \hat{y}_{i}$ has a mean 0 and variance $\sigma^{2}$. ...
- 187
6
votes
1
answer
815
views
Prove that $\frac{(n-2)s^2}{\sigma^2}\sim \chi^{2}_{n-2}$
Consider the following simple linear regression model involving the $\epsilon_i$ error term,
$$y_i = \alpha + \beta x_i + \epsilon_i$$
such that,
$$\epsilon_i \sim \mathcal N(0,\sigma^2)$$
we know ...
- 650
0
votes
1
answer
787
views
Standard error and confidence interval at a point for the fitted value
I have a simple linear regression model $G: Y = \beta_0 + X\beta_1 + \epsilon$. I have found least square estimates for the coefficients, i.e. $\hat\beta_0 = 32.1359$ and $\hat\beta_1=-14.5388$. I ...
- 121
0
votes
1
answer
327
views
A simple proof for expressing $SSR/\sigma^2$ in simple linear regression as the square of a standard normal
I am new to statistics and trying to prove that in simple linear regression, the $SS_{\text{Regression}}/\sigma^2$ can be expressed as the square of a standard normal. ( Where $SS_{\text{Regression}} =...
- 213
0
votes
1
answer
320
views
Least square estimators in the simple linear regression are independent of sum of square of residual [duplicate]
Suppose I have a simple linear regression model:
$$
y_i = \alpha + \beta x_i + \epsilon_i,
$$
and I know the expression for the least square estimators of $\alpha$ and $\beta$:
$$
\hat{\alpha} = \bar{...
- 51