Linked Questions
16 questions linked to/from In simple linear regression, where does the formula for the variance of the residuals come from?
0
votes
1
answer
437
views
Evaluating covariance terms for variance of residual in simple linear regression [duplicate]
I am trying to work through calculating the variance of a residual for simple linear regression not using vectors or matrices. I am having trouble with calculating two different covariance expressions ...
- 101
41
votes
2
answers
39k
views
Understanding shape and calculation of confidence bands in linear regression
I am trying to understand the origin of the curved shaped of confidence bands associated with an OLS linear regression and how it relates to the confidence intervals of the regression parameters (...
- 411
10
votes
1
answer
5k
views
Are the estimates of the intercept and slope in simple linear regression independent?
Consider a linear model
$y_i= \alpha + \beta x_i + \epsilon_i$
and estimates for the slope and intercept $\hat{\alpha}$ and $\hat{\beta}$ using ordinary least squares. This reference for a ...
- 506
5
votes
1
answer
6k
views
Simple Linear Regression: how does $\Sigma\hat{u_i}^2/\sigma^2$ follow chi squared distribution with df (n-2)?
My question is, as far as i am aware,
1. the residuals($\hat{u_i}$) are not independent of one another
2. the variance of ith residual is $\sigma\{(1-1/n-(X_i-\overline{X})/\Sigma(X_i-\overline{X})^2\}...
- 95
0
votes
1
answer
9k
views
Residual variance formulas difference
There is a bi-dimensional table of frequencies:
Doing the regression analysis with the fit formula being $\hat y=a+bx^2$, where $\hat y$ is the same as $y^{est}$, the filled table looks like this:
...
- 139
4
votes
1
answer
3k
views
MSPE formula - is the number of variables not important?
The formula I usually see for MSE is:
$$\mathrm{MSE} = \frac{\sum\limits_{t=1}^T e_i^2}{n-k-1},$$
Whereas for MSPE it is usually:
$$\mathrm{MSPE} = \frac{\sum\limits_{t=T}^{T+P} e_i^2}{P}.$$
So ...
- 973
4
votes
1
answer
3k
views
Deriving SSE of Simple Linear Regression is $\chi^{2}$
As per my notes, the key step in the proof that the sum of squares of residuals in regression is $\chi^{2}_{n-2}$ is the fact that $e_{i} = y_{i} - \hat{y}_{i}$ has a mean 0 and variance $\sigma^{2}$. ...
- 217
3
votes
1
answer
3k
views
"variance of residuals" versus estimated residual variance?
I was instructed on an assignment to "calculate variance of the residuals obtained from your fitted equation." It was a simple linear regression, so I thought "ok, it's just the sum of ...
- 31
4
votes
1
answer
3k
views
SLR: Variance of a residual
I am having problems calculating the variance of a residual in an SLR setting,
ie $\text{var}$$(y_i- \hat{y_i})$. Here is what I have thus far.
If $ \hat{y_i}= \hat{\beta_0} + \hat{\beta_1}x_i$ ...
- 43
3
votes
3
answers
328
views
Demeaning variables in OLS
I am analysing some data in R where I have information on $y$ and $x$.
When I run $y = \alpha + \beta\cdot x$ I get the same coefficient on $x$ (i.e. $\beta$) as ...
- 55
2
votes
2
answers
702
views
help to understand how residual standard deviation can differ at different points on X
I read in more than one place that residual standard deviation can differ at different points on X. I cannot understand this statement.
I find this while learning the very basics, so for me the ...
user110848
4
votes
1
answer
666
views
In simple linear regression, how does the derivation of the variance of the residues support its 'Constant Variance' Assumption?
In simple linear regression:
$$Residuals = \hat{Y} - Y$$
We can derive that:
$$Var(Residuals) = Var(\hat{Y} - Y) = (I-H)\cdot\sigma^2$$ ($\sigma^2$ is the variance of $Y$)
(See derivation of Var(...
- 41
0
votes
0
answers
1k
views
Absolute value of residuals in simple linear regression
In a simple linear regression model
$$E(Y|X=x)=\beta_0+\beta_1x,$$
where the parameters $\beta_0, \beta_1$ are estimated via OLS as
$$\hat{\beta}_1=\frac{\mathrm{Cov}(X,Y)}{\mathrm{Var}(X)}, \text{...
2
votes
1
answer
653
views
Compare the variances of restricted and unrestricted estimators?
Problem
Given a linear model $y_i = \beta_1 + \beta_2 x_i +\epsilon_i, \quad i = 1, \dots, n$
I need to compare the variance ordinary least squares estimator of $\beta_2$ without the restrictions and ...
- 27
0
votes
0
answers
268
views
evaluating out of sample accuracy
I estimate a linear regression model and compute the variance of residuals in both the training-set and also on an additional test set. Ideally these should not be very different.
Does it make sense ...
- 21