Linked Questions

4
votes
0answers
109 views

How are $\hat{\beta}$ and $\hat{\sigma}^2$ independent in OLS when assuming that $\epsilon \sim N(0, \sigma^2)$? [duplicate]

In Elements of Statistical Learning, pg 47, at the very bottom, it states that $\hat{\beta}$ and $\hat{\sigma}^2$ are statistically independent. Is this saying that they are independent when ...
0
votes
1answer
105 views

A simple proof for expressing $SSR/\sigma^2$ in simple linear regression as the square of a standard normal

I am new to statistics and trying to prove that in simple linear regression, the $SS_{\text{Regression}}/\sigma^2$ can be expressed as the square of a standard normal. ( Where $SS_{\text{Regression}} =...
0
votes
1answer
94 views

Least square estimators in the simple linear regression are independent of sum of square of residual

Suppose I have a simple linear regression model: $$ y_i = \alpha + \beta x_i + \epsilon_i, $$ and I know the expression for the least square estimators of $\alpha$ and $\beta$: $$ \hat{\alpha} = \bar{...
5
votes
1answer
451 views

Prove that $\frac{(n-2)s^2}{\sigma^2}\sim \chi^{2}_{n-2}$

Consider the following simple linear regression model involving the $\epsilon_i$ error term, $$y_i = \alpha + \beta x_i + \epsilon_i$$ such that, $$\epsilon_i \sim \mathcal N(0,\sigma^2)$$ we know ...
0
votes
1answer
513 views

Standard error and confidence interval at a point for the fitted value

I have a simple linear regression model $G: Y = \beta_0 + X\beta_1 + \epsilon$. I have found least square estimates for the coefficients, i.e. $\hat\beta_0 = 32.1359$ and $\hat\beta_1=-14.5388$. I ...
3
votes
1answer
2k views

Why error sum of squares has n-2 df (possibly not duplicate, please read on)? (Regression Question Series - Part 4)

In simple linear regression, the error sum of squares is given by $$ \text{SSE} = \sum_{i=1}^n(y_i - \hat{y_i})^2 \\ \hat{\sigma}^2 = s^2 = \dfrac{\text{SSE}}{n-2} $$ where $n-2$ is the degrees of ...
2
votes
2answers
1k views

Joint distribution of least square estimates $(\hat\alpha,\hat\beta)$ in a simple linear regression model [duplicate]

Let $Y_1,Y_2,\ldots,Y_n$ be independently distributed random variables such that $Y_i\sim\mathcal N(\alpha+\beta x_i,\sigma^2)$ for all $i=1,\ldots,n$. If $\hat\alpha$ and $\hat\beta$ be the least ...
2
votes
1answer
2k views

Deriving SSE of Simple Linear Regression is $\chi^{2}$

As per my notes, the key step in the proof that the sum of squares of residuals in regression is $\chi^{2}_{n-2}$ is the fact that $e_{i} = y_{i} - \hat{y}_{i}$ has a mean 0 and variance $\sigma^{2}$. ...
6
votes
2answers
10k views

Why divide RSS by n-2 to get RSE?

$$\text{RSE}=\sqrt{\frac{1}{n-2}\text{RSS}}=\sqrt{\frac{1}{n-2}\sum_{i=1}^n (y_i-\hat{y_i})^2}$$ Context: Simple Linear Regression, an intercept and a slope I have 2 question regarding this issue. ...
26
votes
3answers
33k views

In simple linear regression, where does the formula for the variance of the residuals come from?

According to a text that I'm using, the formula for the variance of the $i^{th}$ residual is given by: $\sigma^2\left ( 1-\frac{1}{n}-\frac{(x_{i}-\overline{x})^2}{S_{xx}} \right )$ I find this hard ...
38
votes
3answers
23k views

Why is RSS distributed chi square times n-p?

I would like to understand why, under the OLS model, the RSS (residual sum of squares) is distributed $$\chi^2\cdot (n-p)$$ ($p$ being the number of parameters in the model, $n$ the number of ...