1
$\begingroup$

Let $X$ and $Y$ be mean 0 and variance 1 random variables such that we choose $\alpha$ and $\beta$ to minimise

$$\mathbb{E}(X-\beta Y)^2$$

and

$$\mathbb{E}(Y-\alpha X)^2$$

after not so difficult derivation, I arrive at $\alpha = \mathbb{E}(XY)/\mathbb{E}X^2$ and $\beta = \mathbb{E}(XY)/\mathbb{E}Y^2$, so $\alpha = \beta$.

This seems very strange, because if $y=mx$ is regression line, then surely $x = \frac{1}{m }y$.

Improve this question
$\endgroup$
11
  • $\begingroup$ (X - Y)^2 = (Y - X)^2 for each X and Y. Basically it's the same function (expand the expressions for prove). That's why you get alpha = beta $\endgroup$
    – Ivan
    Oct 23, 2015 at 12:12
  • $\begingroup$ @Ivan sorry, you are going too fast for me. How are they the same function? The coefficient is in front of $X$ in one of the expression and $Y$ in the other. $\endgroup$
    – Lost1
    Oct 23, 2015 at 12:15
  • $\begingroup$ that's minimization problem. In best case you have E(X - \betaY)^2 = 0 expanding the expression X^2 - 2X\betaY + Y^2 = 0 \beta = (X^2 - 2xy + y^2) and E(Y-\alphaX)^2 = 0 \alpha = (Y^2 - 2xy + X^2) witch is basically the same $\endgroup$
    – Ivan
    Oct 23, 2015 at 12:19
  • $\begingroup$ @Ivan when you expand you should get $X^2+2\beta XY +\beta^2 Y^2$? why has the beta's disappeared? I do not follow this. $\endgroup$
    – Lost1
    Oct 23, 2015 at 12:21
  • 1
    $\begingroup$ What do you precisely mean by "because if $y=mx$ is regression line, then surely $x = \frac{1}{m }y$". Is it to say that, if $\hat\beta$ is the regression coefficient of a regression of $y$ on $x$, $\hat\beta^{-1}$ is the coefficient of the regression of $x$ on $y$? $\endgroup$
    – Christoph Hanck
    Oct 23, 2015 at 14:07

1 Answer 1

Reset to default
0
$\begingroup$

So I will answer my own question. The reason the two regression lines are different is this:

The regression line of $Y$ against $X$ minimises the total squared distance of the $Y$ COORDINATES to the $y$ COORDINATES of the line, where as $X$ against $Y$ minimise the total squared distance of $X$ COORDINATES to the $x$ COORDINATES line.

The confusion I had with regards to the line should be unique comes from the mistaken assumption the regression line minimise the total squared (PERPENDICULAR) distance of each point to the regression line. If this had been the case, my assumption would have been true. However, the two measure of distance never coincide unless there is perfect correlation.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I think @Christoph Hanck's link provides the answer. $\endgroup$
    – mr.rox
    Oct 23, 2015 at 15:47
  • $\begingroup$ @mr.rox well, ive realised before i saw the link. it doesn't matter any more. $\endgroup$
    – Lost1
    Oct 23, 2015 at 15:53

Not the answer you're looking for? Browse other questions tagged or ask your own question.