6
$\begingroup$

I am trying to figure out the meaning of these different "hatted" terms in regression analysis. Here is my basic understanding:

  • $Y$ the original of population/sample values
  • $\hat{Y}$ regression line
  • $\beta$ The coefficient, using population values
  • $\hat{\beta}$ The coefficient, using sample values.

But I have a question about the error term, is it:

  1. $e$ (or $u$) The error term, using the population values, $\hat{e}$ The error term, using sample values (so the error is caused by lack of data)

OR

  1. does $\hat{e}$ simply mean the measurement error (error is caused by measurement)?

Also, is there any way to distinguish $Y$ as the population values and $Y$ as the values of a sample.

$\endgroup$
  • $\begingroup$ Could you please edit your question using LaTeX commands? It is hard to read as is. For instance, use $\hat{Y}$ and $\hat{\beta}$. $\endgroup$ – Xi'an Dec 14 '14 at 14:58
  • $\begingroup$ You should definitely distinguish random variables from observed values (i.e. don't call them both $Y$'s as if they were the same thing). The usual means is to use $Y_i$ for the random variable and $y_i$ for the observed value on that random variable, but as long as it's absolutely clear which object you are talking about, other notation may be reasonable. $\endgroup$ – Glen_b -Reinstate Monica Dec 14 '14 at 23:04
8
$\begingroup$

$e$ is the population error (the part of $Y$ not explained by the linear combination of $X$ and $\beta$), while $\hat{e}$ are the residuals (the sample part of $Y$ not explained by $X$ and $\hat{\beta}$). Linear regression assumes $e \overset{\rm iid}{\sim} N(0, \sigma^2)$ with $\sigma^2$ constant, and we should check this using the observed sample $\hat{e}$.

The error is any source of variation of $Y$ not included in the model, either from excluded variables or from measurement error, as long as they comply with the distributional assumptions.

We usually use $Y$ to refer generically to the population, and $y_i$ to refer to the $i^{\rm th}$ sampled observations.

$\endgroup$
  • $\begingroup$ @mugen Can the population values also have measurement error in them? I thought that they were supposed to describe the population as is (without errors). That would certainly explain. Thanks! $\endgroup$ – Tony Dec 14 '14 at 17:07
  • $\begingroup$ @Tony you never observe the population and hence there's no error to compute, so I'd say measurement errors arise when you observe your sample. $\endgroup$ – mugen Dec 14 '14 at 18:59
  • $\begingroup$ @Tony, in general we believe that there is some level of irreducible randomness, which the error represents. (There are difference metaphysical stances one can take here; some believe that the world is deterministic & that the error is due to the influences of unmeasured causal variables.) If you believe there is measurement error (ie the true value of yi was 12, but someone read off / wrote down 11) then the measurement error gets folded into the residuals e-hat, but isn't considered to be present in the true population errors e. $\endgroup$ – gung - Reinstate Monica Dec 14 '14 at 19:10
  • $\begingroup$ @gung Unless you think of measurement error as a random process - in which case you can certainly talk about a population. $\endgroup$ – Glen_b -Reinstate Monica Dec 14 '14 at 23:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.