0
$\begingroup$

I am trying to understand how to arrive at $r = \dfrac{Cov(X,Y)}{\sigma_X\sigma_Y}$ with a logical narrative. This in fact is kind of continuation from my this unanswered question.

I see that by standardizing the X and Y, the resultant regression line contains $r$ as the slope. But I have to reason that out why should I do that. This is my current narrative.

My narrative:

  1. Covariance is given by below equation which implicitly states its symmetrical nature.

$$ Cov(X,Y) = \sum_x\sum_y(x-\overline{x})(y - \overline{y})p(x,y) = Cov(Y,X) \tag{1} $$

So X covaries with Y as much as Y with X as per above measure.

  1. However, simple regression lines are not symmetric.

$$ \hat{Y}|x = \hat{\beta_0} + \hat{\beta_1}x \ \ , \ \ \text{where} \ \ \ \ \hat{\beta_1} = \dfrac{\sum_i(y_i - \overline{y})(x_i - \overline{x}) }{\sum_i(x_i - \overline{x})^2} \ \ , \ \ \hat{\beta_0} = \overline{y} - \hat{\beta_1}\overline{x} \\ \hat{X}|y = \hat{\beta_2} + \hat{\beta_3}y \ \ , \ \ \text{where} \ \ \ \ \hat{\beta_3} = \dfrac{\sum_i(y_i - \overline{y})(x_i - \overline{x}) }{\sum_i(y_i - \overline{y})^2} \ \ , \ \ \hat{\beta_2} = \overline{x} - \hat{\beta_2}\overline{y} \tag{2} $$

Thus, $\hat{\beta_1} \neq \hat{\beta_2}$.

  1. Given the disadvantage of Covariance being critically dependent on units making it unsuitable to compare different pairs of RVs (or events), we seek a standard measure like Covariance but unitless.

  2. Now by standardizing X and Y, we get new regression lines $\hat{Y}|x, \hat{X}|y$ where the x and y intercepts are zero, and both lines have equal slope which is unitless. That is,

    If I do a full standardization on the sample set,

$$ X_s = \dfrac{X - \overline{X}}{s_X} \ \ , \ \ Y_s = \dfrac{Y - \overline{Y}}{s_Y} $$

we get, with new standardized sample set (i.e $x,y$ now represent new sample set)

$$ \hat{Y_s}|x_s = 0 + \hat{\beta_{1s}}x_s \ \ , \ \ \text{where} \ \ \ \ \hat{\beta_{1s}} = \dfrac{\sum_i(y_{is} - \overline{y_s})(x_{is} - \overline{x_s}) }{\sum_i(x_{is} - \overline{x_s})^2} \ \ \ \ \\ \hat{X_s}|y_s = 0 + \hat{\beta_{3s}}y_s \ \ , \ \ \text{where} \ \ \ \ \hat{\beta_{3s}} = \dfrac{\sum_i(y_{is} - \overline{y_s})(x_{is} - \overline{x_s}) }{\sum_i(y_{is} - \overline{y_s})^2} \ \ \ \ \tag{3} $$

results in

$$ r = \hat{\beta_{1s}} = \hat{\beta_{3s}} \tag{4} $$

that is, the regression lines are symmetric to each other.

  1. In fact, reversing this procedure to non standardized raw X and Y, we could say their regression lines have relation with correlation as below

$$ r = \hat{\beta_1}\dfrac{s_X}{s_Y} = \hat{\beta_3}\dfrac{s_Y}{s_X} \tag{5} $$

My questions:
1. Is my above narrative correct and minimally complete? What went wrong? What could be added? How could I improvise?
2. I see, Galton discovered regression via a bivariate normal distribution link . How did we then generalized it to any or random distribution?
3. Also a perfect linearity would mean, underlying distribution is bivariate normal?
4. After this narrative, how could I prove this sample $r$ applies to population $\rho$ also?
5. I hope to see final $r$ equalling cosine product of standardized dot product also. That is,

$$ r = cos\theta = \dfrac{(x - \overline{x})\bullet(y - \overline{y})}{\lvert x - \overline{x} \rvert \lvert y - \overline{y} \rvert} \tag{6} $$

Then, what would unstandardized dot product refer to or how related to non standardized equation set (2)? That is $$ cos\theta = \dfrac{x\bullet y}{\lvert x \rvert \lvert y \rvert} = ? \tag{7} $$

$\endgroup$
  • 1
    $\begingroup$ At first, you need to understand the difference between the parameters and the estimate of them, then make the equations meaningful. One example, $E(Y|x) = \hat{\beta_0} + \hat{\beta_1}x$. It does not follow the popular rules. $E(Y|x) = {\beta_0} + {\beta_1}x$ and $(\hat Y|x) = \hat{\beta_0} + \hat{\beta_1}x$ are correct, but they have different meaning. $\endgroup$ – user158565 Nov 11 '18 at 15:46
  • $\begingroup$ oh thanks for pointing that out. I just updated. Is it now ok? $\endgroup$ – Parthiban Rajendran Nov 11 '18 at 16:15
  • $\begingroup$ and looking forward for your answer on the question $\endgroup$ – Parthiban Rajendran Nov 11 '18 at 16:16
  • 1
    $\begingroup$ Not OK. I just gave you an example, still have problems. I have no answer to your question, because I do not know what you asked for. $\endgroup$ – user158565 Nov 11 '18 at 16:28
  • $\begingroup$ ok can you at least list out all problems so I could update? I am not able to see any or that that hinders understanding my question at least. Is it only with the equations? $\endgroup$ – Parthiban Rajendran Nov 11 '18 at 16:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.