1
$\begingroup$

Using simple linear regression model, and sample correlation coefficient $r$, for a sample set $X,Y$, the true regression line could be given as below.

$$ \hat{Y}|x = \overline{y} + r\dfrac{s_Y}{s_X}(x - \overline{x}) \tag{1} $$

This was achieved, as per my understanding, without assuming any particular distribution of X and Y (only error $\varepsilon$ was assumed to be $N(0, s^2)$).

Now $\hat{Y}|x$ is mean estimate of random variable $Y|x$. So this RV should have a variance. As per Devore's book, here, (page 473,section 12.1), the variance is same as error assumed.

$$ s_{Y|x}^2 = V(\beta_0 + \beta_1x + \varepsilon) = Var(\varepsilon) = s^2 \tag{2} $$

An online statistics book here calculates a seemingly different formula (I have corrected to differentiate for sample and population)

$$ s_{Y|x}^2 = s_Y^2(1 - r) \tag{3} $$

My questions:

  1. Is both (2) and (3) equivalent, and just in different terms? If not, when each holds true?
  2. Till (2) no distribution of X and Y was assumed. (3) assumes normal distribution of both X and Y. Is it necessary?
  3. Can I extend this (3) directly to population? I get confused here as there is usually nothing like regression line for population, but only for given sample sets.
$\endgroup$
  • $\begingroup$ Your second equation simply represents conditional variance of $Y$ given $x$ as constant, whereas, third equation represents the same in terms of unconditional variance of $Y$. More specifically, you can say, the 3rd equation shows the relationship between unconditional and conditional variance of $Y$. $\endgroup$ – Neeraj Nov 17 '18 at 8:42
0
$\begingroup$

Taking your questions one by one:

  1. Is both (2) and (3) equivalent, and just in different terms? If not, when each holds true?

Answer is already provided in the comment. Your second equation simply represents conditional variance of $Y$ given $x$ as constant, whereas, third equation represents the same in terms of unconditional variance of $Y$. More specifically, you can say, the 3rd equation shows the relationship between unconditional and conditional variance of $Y$.

  1. Till (2) no distribution of X and Y was assumed. (3) assumes normal distribution of both X and Y. Is it necessary?

Neither equation 2, nor equation 3 does not make any explicit assumption about the specific distribution regarding $X$ and $Y$. Your link that shows the derivation of equation 3, does not make use of normal distribution anywhere. So, it holds true in general. (Provided your equation one is not based on assumption of normal distribution in population context)

  1. Can I extend this (3) directly to population? I get confused here as there is usually nothing like regression line for population, but only for given sample sets.

These results are derived from population regression function, not sample regression function. The underlying equation for these results is PRF.

$\endgroup$
  • $\begingroup$ So for (1), I take you say that it is equivalent? Because this would mean, $s_Y^2 = \dfrac{s^2}{1 - r}$. During perfect correlation, this leads to $s_Y^2 = \infty$? $\endgroup$ – Parthiban Rajendran Nov 17 '18 at 9:13
  • $\begingroup$ for (3), here and here are the snapshots of their post making normal assumption. In fact in each page, they emphasize their assumptions. but yes, equation derivation is not making use of normal distribution anyway, this is why I wanted to confirm if (3) applies to any distribution, not just normal. $\endgroup$ – Parthiban Rajendran Nov 17 '18 at 9:18
  • $\begingroup$ Just think, if there is perfect correlation between $X$ and $Y$, then we do not need any error terms in PRF. The function can be written as $Y = \beta_0 + \beta_1 x$. In this case, the conditional variance of $Y$ given $X$ will become zero. Exactly shown by equation 3. $\endgroup$ – Neeraj Nov 17 '18 at 9:19
  • $\begingroup$ what is a PRF? Probability Regression Function? (SE stresses 15 chars in comment length) $\endgroup$ – Parthiban Rajendran Nov 17 '18 at 9:21
  • $\begingroup$ Since, you are talking about regression function, therefore, it is population regression function. $\endgroup$ – Neeraj Nov 17 '18 at 9:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.