Linked Questions
25 questions linked to/from Why is RSS distributed chi square times n-p?
1
vote
0
answers
9k
views
Linear regression: Unbiased estimator of the variance of outputs [duplicate]
I'm having trouble understanding something from the linear regression chapter of Elements of Statistical Learning.
We have a fixed $N\times p$ matrix $\mathbf{X}$ ($N$ inputs with $p$ predictors)
...
- 283
1
vote
1
answer
3k
views
Residual Sum of Squares degrees of freedom intuition [duplicate]
Let RSS = Residual sum of squares $ = \sum (y_i - \hat{y}_i)^2$. Without proof, $\frac{RSS}{\sigma^2} \sim \chi^2_{n-2}$. I do not quite understand why the DoF is $n-2.$ Could someone explain?
- 153
1
vote
1
answer
979
views
Proof of the distribution of the residual standard error [duplicate]
In my notes from university I have written down that the residual standard error (from normal linear regression) has the following distribution
$\frac{\hat{\sigma}^2}{\sigma^2}\sim \frac{\chi^{2}_{n-...
- 790
0
votes
0
answers
256
views
Understanding sum of square deviations [duplicate]
Given $X_1...X_n\stackrel{iid}{\sim} N(\mu,\sigma^2)$ and $U=\sum_{i=1}^n (X_i-\overline{X})^2$,
why is $U\sim\sigma^2 \chi_{n-1}^2$ ?
And what would be the distribution of $V=\sum_{i=1}^n (X_i-\mu)^...
- 211
33
votes
3
answers
15k
views
Prove that F-statistic follows F-distribution
In light of this question : Proof that the coefficients in an OLS model follow a t-distribution with (n-k) degrees of freedom
I would love to understand why
$$ F = \frac{(\text{TSS}-\text{RSS})/(p-1)...
- 433
39
votes
1
answer
25k
views
Covariance of a random vector after a linear transformation
If $\mathbf {Z}$ is random vector and $A$ is a fixed matrix, could someone explain why $$\mathrm{cov}[A \mathbf {Z}]= A \mathrm{cov}[\mathbf {Z}]A^\top.$$
- 745
30
votes
2
answers
23k
views
Why is a T distribution used for hypothesis testing a linear regression coefficient?
In practice, using a standard T-test to check the significance of a linear regression coefficient is common practice. The mechanics of the calculation make sense to me.
Why is it that the T-...
- 403
14
votes
5
answers
15k
views
Why divide by $n-2$ for residual standard errors
I was just watching a lecture on statistics and someone was calculating something called the residual standard error. It looked a lot like finding the average of the square of the residuals, the ...
- 259
14
votes
1
answer
17k
views
Distribution of sum of squares error for linear regression?
I know that distribution of sample variance
$$
\sum\frac{(X_i-\bar{X})^2}{\sigma^2}\sim \chi^2_{(n-1)}
$$
$$
\sum\frac{(X_i-\bar{X})^2}{n-1}\sim \frac{\sigma^2}{n-1}\chi^2_{(n-1)}
$$
It's from the ...
- 1,231
14
votes
3
answers
1k
views
Easy proof of $\sum_{i=1}^n \left(Z_i - \bar{Z}\right)^2 \sim \chi^2_{n-1}$?
Let $Z_1,\cdots,Z_n$ be independent standard normal random variables. There are many (lengthy) proofs out there, showing that
$$ \sum_{i=1}^n \left(Z_i - \frac{1}{n}\sum_{j=1}^n Z_j \right)^2 \sim \...
- 1,696
9
votes
2
answers
726
views
Standard Error of Noise Variance in Least Squares
Suppose I'm doing ordinary least squares with homoskedastic errors, so something like:
$$
y = X \beta + \epsilon
$$
where $\epsilon \sim \mathcal{N}(0,\sigma^2)$.
I know how to estimate the expected ...
- 255
4
votes
1
answer
12k
views
Proof that regression residual error is an unbiased estimate of error variance
Consider the least squares problem $Y=X\beta +\epsilon$ while $\epsilon$ is zero mean Gaussian with $E(\epsilon) = 0$ and variance $\sigma^2$. I need to prove that
$\frac{V(\hat{\beta})}{N-(n+m)}$ ...
- 43
6
votes
1
answer
8k
views
Simple Linear Regression: how does $\Sigma\hat{u_i}^2/\sigma^2$ follow chi squared distribution with df (n-2)?
My question is, as far as i am aware,
1. the residuals($\hat{u_i}$) are not independent of one another
2. the variance of ith residual is $\sigma\{(1-1/n-(X_i-\overline{X})/\Sigma(X_i-\overline{X})^2\}...
- 105
5
votes
1
answer
18k
views
What is "estimated unbiased variance of the error term"?
Disclosure: This is a homework question.
I have fit a multiple linear regression model in eviews, and I am asked to calculate "estimated unbiased variance of the error term, i.e., $\hat\sigma^2$".
...
- 173
4
votes
1
answer
4k
views
Deriving SSE of Simple Linear Regression is $\chi^{2}$
As per my notes, the key step in the proof that the sum of squares of residuals in regression is $\chi^{2}_{n-2}$ is the fact that $e_{i} = y_{i} - \hat{y}_{i}$ has a mean 0 and variance $\sigma^{2}$. ...
- 257
6
votes
1
answer
2k
views
Prove that $\frac{(n-2)s^2}{\sigma^2}\sim \chi^{2}_{n-2}$
Consider the following simple linear regression model involving the $\epsilon_i$ error term,
$$y_i = \alpha + \beta x_i + \epsilon_i$$
such that,
$$\epsilon_i \sim \mathcal N(0,\sigma^2)$$
we know ...
- 730
2
votes
1
answer
2k
views
Why is the MLE for variance in single linear regression biased? [duplicate]
I understand that the Maximum Likelihood Estimator for variance, in general, is biased (the average calculated from the sample itself reduces the degree of freedom by 1 e.t.c):
...
- 143
4
votes
1
answer
2k
views
Why use the F distribution and F test?
I don't understand why in the F test we calculate the ratio between MSE between subject and MSE within subject. As far as I know, this is due because we want to use the F distribution, which is a rate ...
- 587
2
votes
0
answers
1k
views
Distribution of the residual sum of squares for *multivariate* linear regression
I want to ask about a multivariate generalization of this previous question.
It is a well established fact that in univariate (i.e. the response $y$ is univariate) linear regression, that the ...
- 207
2
votes
1
answer
954
views
Why is the expected value of variance different than the expected value of Maximum Likelihood variance?
The expected value of variance is:
The expected value of the maximum likelihood of variance is:
$E\left[\frac{1}{N}\sum_{n=1}^{N}(X-\mu_{ml})^{2})\right] = \frac{N-1}{N}\sigma ^{2} $
Why does ...
- 341
2
votes
2
answers
173
views
Computing variance from a set of samples
My dataset contains a set of samples from a set of normal RVs. Each RV is normally distributed with equal variances and varying means. However, I have only two samples from each RV.
How to estimate ...
1
vote
1
answer
67
views
Independence statement on an LSE theorem
I am studying some properties of the least squares estimator and I have the following statement, with $X$ a full rank matrix
If $e$ is independent of $\hat y = X \hat \beta$, then $S^2=e^Te/(n-p)$ ...
- 345
1
vote
0
answers
83
views
How to find $\mathbb{E} \left[\frac{\bar{\mu}}{\bar{\sigma}^2}\right]$?
I asked the same question on math stacks: MathStacks:, and some user suggest to ask it here for better insight. So this question has found interest in many research problems, but there have been no ...
- 11
1
vote
0
answers
72
views
Variance of $\hat{\sigma}^2=\frac{1}{n}\sum_{i=1}^n(Y_i - \hat{Y}_i)^2$ in regression
Consider a regression problem
$Y_i = X_i'\beta+e_i$ with $\beta \in \mathbb{R}^p$. $e_i$'s are i.i.d. with $E(e)=0$ and $Var(e)=\sigma^2<\infty$.
My question is about the (asymptotic) variance ...
- 11
0
votes
0
answers
26
views
Regression distribution of Residuals - Chi square test
I'm trying to solve a conundrum but can't figure out where I'm making the mistake.
Let me define the Linear model as
$$
\begin{align}
y &= X \beta + \epsilon \\
\end{align}
$$
Where $...
- 153