I am reading some nice lecture notes on basic statistics, learning some basic topics on estimation of parameters. Reading the chapter about the method of least squares estimation I meet the following phrase: "Consider the situation in which the $Y_i$ from the random sample can be written in the form $$Y_i=\beta+\epsilon_i=\hat\beta+e_i,$$
where $\mathbb{E}[\epsilon_i]=0$ and $\mathrm{Var}(\epsilon_i)=\sigma^2$ for all $i$".
Could you please explain the difference between $e_i$ and $\epsilon_i$? Why does the author replace $e_i$ by $\epsilon_i$ and $\beta$ by $\hat\beta$? What does he mean?
$\begingroup$
$\endgroup$
1
-
2$\begingroup$ (+1) Although you could not be expected to find the thread at stats.stackexchange.com/questions/7581 in this context, it does answer your question. The connection is that your lecture notes must assume you understand $\beta$ to be a parameter and $\hat\beta$ to be an estimate of that parameter. $\endgroup$– whuber ♦Commented May 18, 2015 at 15:33
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
This is statistics nomenclature. You can just deal with it frontally, or inject some poetry into it - I vote for the latter.
Some say mathematicians (and scientist possibly by extension) are Platonic. And this is certainly true of statistics in its conceptual underpinnings. Much like in the allegory of the cave, the population concept in statistics inhabits the ethereal world of Ideas. In a sense, it is unknowable, but we can infer what it is like by paying close attention to the shadows on the walls of the cave, or at a more prosaic level, by sampling.
Now, and really answering your question, everything pure, and epistemologically unreachable - again, the population parameters - is paid due respect by using the Greek alphabet. Hence, $\epsilon$, above, would be the error term if we could gather information of all and every single subject of the entire population, plot the measurements for each one of them, and draw a best fitting line, which would have a slope, you guessed it, $\beta$.
Of course the equal sign that follows in the first equation in your question, is pure utopia (please double check), because you can't ever reach these idealized parameters, and will have to content yourself with the estimates from the sample, which are accordingly diminished through the use of the Latin alphabet (i.e. "$e$") and/or by placing a humble hat on top of the Greek symbol (i.e. $\hat{\beta}$).
answered May 18, 2015 at 15:37