Linked Questions

0
votes
0answers
5k views

Linear regression: Unbiased estimator of the variance of outputs [duplicate]

I'm having trouble understanding something from the linear regression chapter of Elements of Statistical Learning. We have a fixed $N\times p$ matrix $\mathbf{X}$ ($N$ inputs with $p$ predictors) ...
1
vote
1answer
584 views

Residual Sum of Squares degrees of freedom intuition [duplicate]

Let RSS = Residual sum of squares $ = \sum (y_i - \hat{y}_i)^2$. Without proof, $\frac{RSS}{\sigma^2} \sim \chi^2_{n-2}$. I do not quite understand why the DoF is $n-2.$ Could someone explain?
1
vote
1answer
647 views

Proof of the distribution of the residual standard error [duplicate]

In my notes from university I have written down that the residual standard error (from normal linear regression) has the following distribution $\frac{\hat{\sigma}^2}{\sigma^2}\sim \frac{\chi^{2}_{n-...
0
votes
0answers
66 views

Understanding sum of square deviations [duplicate]

Given $X_1...X_n\stackrel{iid}{\sim} N(\mu,\sigma^2)$ and $U=\sum_{i=1}^n (X_i-\overline{X})^2$, why is $U\sim\sigma^2 \chi_{n-1}^2$ ? And what would be the distribution of $V=\sum_{i=1}^n (X_i-\mu)^...
0
votes
0answers
15 views

Proof of $\chi^2$ statistic used for error variance hypothesis testing [duplicate]

I am trying to find a proof for the following, related to the hypothesis testing of the variance of the error in linear regression: How the following statistic is derived: $$\chi^2= \left( n - k \...
18
votes
2answers
9k views

Why is a T distribution used for hypothesis testing a linear regression coefficient?

In practice, using a standard T-test to check the significance of a linear regression coefficient is common practice. The mechanics of the calculation make sense to me. Why is it that the T-...
21
votes
2answers
7k views

Proof that F-statistic follows F-distribution

In light of this question : Proof that the coefficients in an OLS model follow a t-distribution with (n-k) degrees of freedom I would love to understand why $$ F = \frac{(\text{TSS}-\text{RSS})/(p-1)...
12
votes
1answer
12k views

Distribution of sum of squares error for linear regression?

I know that distribution of sample variance $$ \sum\frac{(X_i-\bar{X})^2}{\sigma^2}\sim \chi^2_{(n-1)} $$ $$ \sum\frac{(X_i-\bar{X})^2}{n-1}\sim \frac{\sigma^2}{n-1}\chi^2_{(n-1)} $$ It's from the ...
4
votes
4answers
6k views

Why divide by $n-2$ for residual standard errors

I was just watching a lecture on statistics and someone was calculating something called the residual standard error. It looked a lot like finding the average of the square of the residuals, the ...
10
votes
2answers
549 views

Easy proof of $\sum_{i=1}^n \left(Z_i - \bar{Z}\right)^2 \sim \chi^2_{n-1}$?

Let $Z_1,\cdots,Z_n$ be independent standard normal random variables. There are many (lengthy) proofs out there, showing that $$ \sum_{i=1}^n \left(Z_i - \frac{1}{n}\sum_{j=1}^n Z_j \right)^2 \sim \...
5
votes
1answer
12k views

What is “estimated unbiased variance of the error term”?

Disclosure: This is a homework question. I have fit a multiple linear regression model in eviews, and I am asked to calculate "estimated unbiased variance of the error term, i.e., $\hat\sigma^2$". ...
3
votes
1answer
7k views

Proof that regression residual error is an unbiased estimate of error variance

Consider the least squares problem $Y=X\beta +\epsilon$ while $\epsilon$ is zero mean Gaussian with $E(\epsilon) = 0$ and variance $\sigma^2$. I need to prove that $\frac{V(\hat{\beta})}{N-(n+m)}$ ...
4
votes
1answer
2k views

Why use the F distribution and F test?

I don't understand why in the F test we calculate the ratio between MSE between subject and MSE within subject. As far as I know, this is due because we want to use the F distribution, which is a rate ...
2
votes
1answer
1k views

Deriving SSE of Simple Linear Regression is $\chi^{2}$

As per my notes, the key step in the proof that the sum of squares of residuals in regression is $\chi^{2}_{n-2}$ is the fact that $e_{i} = y_{i} - \hat{y}_{i}$ has a mean 0 and variance $\sigma^{2}$. ...
2
votes
1answer
1k views

Simple Linear Regression: how does $\Sigma\hat{u_i}^2/\sigma^2$ follow chi squared distribution with df (n-2)?

My question is, as far as i am aware, 1. the residuals($\hat{u_i}$) are not independent of one another 2. the variance of ith residual is $\sigma\{(1-1/n-(X_i-\overline{X})/\Sigma(X_i-\overline{X})^2\}...

15 30 50 per page