# What is MS Residual, and how it differs from the common aleatory error?

Hello I have two questions. I am trying to understand this formulas. But I cant understand how the MS Residual is different from the common aleatory error Another thing, I just labeled each value with the with their supposed corresponding formula, it is correct?

• What is "common aleatory error"? Feb 8 at 19:58
• $MS_{residual}$ probably means "Mean Squared Residual" and doesn't refer to any specific error (hence the "approx" in the last line.) Feb 8 at 20:04

Since the OLS model assumptions are that $$y_i=\beta^\top x_i + \epsilon_i$$ with $$\epsilon_i\stackrel{\text{i.i.d.}}{\sim}\mathcal N(0,\sigma^2)$$, $$\text{MS}_{\text{residual}}:=\frac 1 n \sum_i(y_i-\hat{\beta}^\top x_i)^2$$ is (only) an approximation of the variance $$\sigma^2$$ for two reasons

#### 1. $$\hat \beta\approx\beta\in\mathbb{R}^d$$ (holds in expectation)

The OLS estimator $$\hat\beta$$ is unbiased as $$\mathbb{E}\left[\hat \beta\right]=\beta$$ and the expected error converges to 0 in the limit since $$\mathbb{E}\left[||\hat\beta-\beta||^2\right]=\frac {\sigma^2d}{n}$$

#### 2. $$\frac 1 n \sum_i(y_i-\beta^\top x_i)^2=\frac 1 n \sum_i\epsilon_i^2 \approx\sigma^2$$ (holds in limit)

By the law of large numbers, as $$n \rightarrow \infty$$ we have $$\frac 1 n\sum_i\epsilon_i^2\rightarrow\mathbb{E}\left[\epsilon_i^2\right]=\sigma^2$$

Search "Statistical guarantees for ordinary least squares" for proofs and such

• Nice answer! It would be great if you could specify a bit better "holds in expectation" and "holds in limit" as well as adding more complete infos about your reference to that paper (I guess it's a paper). Thank you Feb 9 at 5:11
• These are very standard undergrad level results, as such I do not think you need/would be able to find a paper, but maybe some free course material. You could easily prove them yourself or find them online. But alright, I will include them here directly. Feb 9 at 9:00
• Someone asking that kind of question likely is not familiar with terms like "holds in expectation". Also, the way you reported Statistical guarantees for ordinary least square it sounded like the title of an article or book. I think it just needed to be easier to follow for a beginner. Feb 10 at 1:13
• Thank you for your time ! Feb 10 at 16:48