Linked Questions

0
votes
1answer
272 views

Evaluating covariance terms for variance of residual in simple linear regression [duplicate]

I am trying to work through calculating the variance of a residual for simple linear regression not using vectors or matrices. I am having trouble with calculating two different covariance expressions ...
39
votes
2answers
33k views

Understanding shape and calculation of confidence bands in linear regression

I am trying to understand the origin of the curved shaped of confidence bands associated with an OLS linear regression and how it relates to the confidence intervals of the regression parameters (...
10
votes
1answer
4k views

Are the estimates of the intercept and slope in simple linear regression independent?

Consider a linear model $y_i= \alpha + \beta x_i + \epsilon_i$ and estimates for the slope and intercept $\hat{\alpha}$ and $\hat{\beta}$ using ordinary least squares. This reference for a ...
0
votes
1answer
9k views

Residual variance formulas difference

There is a bi-dimensional table of frequencies: Doing the regression analysis with the fit formula being $\hat y=a+bx^2$, where $\hat y$ is the same as $y^{est}$, the filled table looks like this: ...
3
votes
1answer
3k views

Simple Linear Regression: how does $\Sigma\hat{u_i}^2/\sigma^2$ follow chi squared distribution with df (n-2)?

My question is, as far as i am aware, 1. the residuals($\hat{u_i}$) are not independent of one another 2. the variance of ith residual is $\sigma\{(1-1/n-(X_i-\overline{X})/\Sigma(X_i-\overline{X})^2\}...
3
votes
1answer
2k views

MSPE formula - is the number of variables not important?

The formula I usually see for MSE is: $$\mathrm{MSE} = \frac{\sum\limits_{t=1}^T e_i^2}{n-k-1},$$ Whereas for MSPE it is usually: $$\mathrm{MSPE} = \frac{\sum\limits_{t=T}^{T+P} e_i^2}{P}.$$ So ...
4
votes
1answer
2k views

SLR: Variance of a residual

I am having problems calculating the variance of a residual in an SLR setting, ie $\text{var}$$(y_i- \hat{y_i})$. Here is what I have thus far. If $ \hat{y_i}= \hat{\beta_0} + \hat{\beta_1}x_i$ ...
2
votes
1answer
2k views

Deriving SSE of Simple Linear Regression is $\chi^{2}$

As per my notes, the key step in the proof that the sum of squares of residuals in regression is $\chi^{2}_{n-2}$ is the fact that $e_{i} = y_{i} - \hat{y}_{i}$ has a mean 0 and variance $\sigma^{2}$. ...
2
votes
2answers
649 views

help to understand how residual standard deviation can differ at different points on X

I read in more than one place that residual standard deviation can differ at different points on X. I cannot understand this statement. I find this while learning the very basics, so for me the ...
2
votes
1answer
1k views

"variance of residuals" versus estimated residual variance?

I was instructed on an assignment to "calculate variance of the residuals obtained from your fitted equation." It was a simple linear regression, so I thought "ok, it's just the sum of ...
3
votes
1answer
417 views

In simple linear regression, how does the derivation of the variance of the residues support its 'Constant Variance' Assumption?

In simple linear regression: $$Residuals = \hat{Y} - Y$$ We can derive that: $$Var(Residuals) = Var(\hat{Y} - Y) = (I-H)\cdot\sigma^2$$ ($\sigma^2$ is the variance of $Y$) (See derivation of Var(...
0
votes
0answers
583 views

Absolute value of residuals in simple linear regression

In a simple linear regression model $$E(Y|X=x)=\beta_0+\beta_1x,$$ where the parameters $\beta_0, \beta_1$ are estimated via OLS as $$\hat{\beta}_1=\frac{\mathrm{Cov}(X,Y)}{\mathrm{Var}(X)}, \text{...
0
votes
0answers
219 views

evaluating out of sample accuracy

I estimate a linear regression model and compute the variance of residuals in both the training-set and also on an additional test set. Ideally these should not be very different. Does it make sense ...
2
votes
1answer
170 views

Compare the variances of restricted and unrestricted estimators?

Problem Given a linear model $y_i = \beta_1 + \beta_2 x_i +\epsilon_i, \quad i = 1, \dots, n$ I need to compare the variance ordinary least squares estimator of $\beta_2$ without the restrictions and ...
0
votes
0answers
51 views

Proving that $V(\hat{y}_{x_0}) = \sigma^2\bigg[\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}\bigg]$ [duplicate]

Exercise : Prove that the variance of $\hat{y}_{x_0} = \hat{b_0} + \hat{b_1}x_0$ is : $$\text{Var}(\hat{y}_{x_0}) = \frac{\sigma^2\sum x_i^2}{n\sum(x_i-\bar{x})^2}+\frac{\sigma^2x_0^2}{\sum(x_i-\...