Linked Questions

1 vote
0 answers
7k views

Linear regression: Unbiased estimator of the variance of outputs [duplicate]

I'm having trouble understanding something from the linear regression chapter of Elements of Statistical Learning. We have a fixed $N\times p$ matrix $\mathbf{X}$ ($N$ inputs with $p$ predictors) ...
1 vote
1 answer
2k views

Residual Sum of Squares degrees of freedom intuition [duplicate]

Let RSS = Residual sum of squares $ = \sum (y_i - \hat{y}_i)^2$. Without proof, $\frac{RSS}{\sigma^2} \sim \chi^2_{n-2}$. I do not quite understand why the DoF is $n-2.$ Could someone explain?
1 vote
1 answer
892 views

Proof of the distribution of the residual standard error [duplicate]

In my notes from university I have written down that the residual standard error (from normal linear regression) has the following distribution $\frac{\hat{\sigma}^2}{\sigma^2}\sim \frac{\chi^{2}_{n-...
  • 594
0 votes
0 answers
179 views

Understanding sum of square deviations [duplicate]

Given $X_1...X_n\stackrel{iid}{\sim} N(\mu,\sigma^2)$ and $U=\sum_{i=1}^n (X_i-\overline{X})^2$, why is $U\sim\sigma^2 \chi_{n-1}^2$ ? And what would be the distribution of $V=\sum_{i=1}^n (X_i-\mu)^...
23 votes
2 answers
18k views

Why is a T distribution used for hypothesis testing a linear regression coefficient?

In practice, using a standard T-test to check the significance of a linear regression coefficient is common practice. The mechanics of the calculation make sense to me. Why is it that the T-...
28 votes
2 answers
11k views

Proof that F-statistic follows F-distribution

In light of this question : Proof that the coefficients in an OLS model follow a t-distribution with (n-k) degrees of freedom I would love to understand why $$ F = \frac{(\text{TSS}-\text{RSS})/(p-1)...
9 votes
4 answers
11k views

Why divide by $n-2$ for residual standard errors

I was just watching a lecture on statistics and someone was calculating something called the residual standard error. It looked a lot like finding the average of the square of the residuals, the ...
14 votes
1 answer
16k views

Distribution of sum of squares error for linear regression?

I know that distribution of sample variance $$ \sum\frac{(X_i-\bar{X})^2}{\sigma^2}\sim \chi^2_{(n-1)} $$ $$ \sum\frac{(X_i-\bar{X})^2}{n-1}\sim \frac{\sigma^2}{n-1}\chi^2_{(n-1)} $$ It's from the ...
  • 1,171
12 votes
2 answers
793 views

Easy proof of $\sum_{i=1}^n \left(Z_i - \bar{Z}\right)^2 \sim \chi^2_{n-1}$?

Let $Z_1,\cdots,Z_n$ be independent standard normal random variables. There are many (lengthy) proofs out there, showing that $$ \sum_{i=1}^n \left(Z_i - \frac{1}{n}\sum_{j=1}^n Z_j \right)^2 \sim \...
  • 1,416
4 votes
1 answer
10k views

Proof that regression residual error is an unbiased estimate of error variance

Consider the least squares problem $Y=X\beta +\epsilon$ while $\epsilon$ is zero mean Gaussian with $E(\epsilon) = 0$ and variance $\sigma^2$. I need to prove that $\frac{V(\hat{\beta})}{N-(n+m)}$ ...
5 votes
1 answer
16k views

What is "estimated unbiased variance of the error term"?

Disclosure: This is a homework question. I have fit a multiple linear regression model in eviews, and I am asked to calculate "estimated unbiased variance of the error term, i.e., $\hat\sigma^2$". ...
  • 173
5 votes
1 answer
5k views

Simple Linear Regression: how does $\Sigma\hat{u_i}^2/\sigma^2$ follow chi squared distribution with df (n-2)?

My question is, as far as i am aware, 1. the residuals($\hat{u_i}$) are not independent of one another 2. the variance of ith residual is $\sigma\{(1-1/n-(X_i-\overline{X})/\Sigma(X_i-\overline{X})^2\}...
3 votes
1 answer
3k views

Deriving SSE of Simple Linear Regression is $\chi^{2}$

As per my notes, the key step in the proof that the sum of squares of residuals in regression is $\chi^{2}_{n-2}$ is the fact that $e_{i} = y_{i} - \hat{y}_{i}$ has a mean 0 and variance $\sigma^{2}$. ...
  • 187
6 votes
1 answer
920 views

Prove that $\frac{(n-2)s^2}{\sigma^2}\sim \chi^{2}_{n-2}$

Consider the following simple linear regression model involving the $\epsilon_i$ error term, $$y_i = \alpha + \beta x_i + \epsilon_i$$ such that, $$\epsilon_i \sim \mathcal N(0,\sigma^2)$$ we know ...
4 votes
1 answer
2k views

Why use the F distribution and F test?

I don't understand why in the F test we calculate the ratio between MSE between subject and MSE within subject. As far as I know, this is due because we want to use the F distribution, which is a rate ...
  • 525

15 30 50 per page