I am given regression equations, one showing linear regression of x on y, and the other y on x. Both refer to the same set of data-points.

4X - 5Y + 33 = 0
20X - 9Y - 107 = 0
  • Taking the first to be X (dependent) on Y (independent) => of the form X = a + bY. I end up with slope value of 1.25

  • Taking the second to be Y (dependent) on X (independent) => of the form Y = a + bX. I end up with slope value of 2.22

Now, these values somehow imply that the system of regression lines is invalid? How is that? I am unable to visualize this.

The book states the "rule" being that:

  1. Both coefficients (slopes) must be less than 1
  2. Both coefficients (slopes) must be of the same sign

Are these correct? What other constraints exist for a set of regression equations to be valid?

  • $\begingroup$ "I am told both regression coefficients cannot exceed 1." -- don't believe everything you're told. Were they perhaps talking about standardized coefficients? $\endgroup$
    – Glen_b
    Aug 31 '13 at 9:45
  • $\begingroup$ The book I am studying says "But this is not possible because both the regression coefficients are greater than 1. Hence, we will now treat eq.1 as Y on X and eq.2 as X on Y. " $\endgroup$ Aug 31 '13 at 9:47
  • 1
    $\begingroup$ This seems rather strange. Which book? Are you sure they're not talking about standardized coefficients? $\endgroup$
    – Glen_b
    Aug 31 '13 at 9:50
  • $\begingroup$ Introductory Statistical Methods by S.P. Gupta (not the most popular of textbooks). I don't know what you mean by standardized coeffs. If that implies the Y-intercept on the lines is 0, it isn't so. $\endgroup$ Aug 31 '13 at 9:54
  • $\begingroup$ Either the book is wrong, or you've misunderstood it. Post the complete text relating to your question. $\endgroup$
    – Hong Ooi
    Aug 31 '13 at 10:01

I am told both regression coefficients cannot exceed 1

Well, yes they can. There are no constraints about values of coefficients. Where did you get it from? The slope is simply the tangent of the angle between your line and OX axis, so it can get any value in $\mathbb{R}$

The only trivial "constraint" is that given simple linear regression from $\mathbb{R}$ to $\mathbb{R}$, if you look at the equations $$ y=ax+b$$ and $$ x=cy+e$$ Then at least one of the $\{a,c\}$ has an absolute value not greater then $1$.

Assuming that $a\neq 0$ $$y=ax+b \iff -ax=b-y \iff x=\frac{-y}{-a}+\frac{b}{-a} \iff x = \frac{y}{a} - \frac{b}{a} \iff x = \frac{1}{a}y - \frac{b}{a}$$

so $c=\frac{1}{a}$, and as a result, if $|a|>1$ then $|c|=\left |\frac{1}{a}\right | < 1$

  • $\begingroup$ Shouldn't there be some constraint on the slopes of regression lines y on x and vice versa? One is a reflection of the other about the x = y line is it not? $\endgroup$ Aug 31 '13 at 9:44
  • $\begingroup$ But what do you mean by "regression lines y on x and vice versa"? You mean running regression (what kind of? OLS?) once with X as input and Y as output and once the other way around, or you mean expressing the simple linear equation once in the form $y=ax+b$ and once in $x=cy+d$? $\endgroup$
    – lejlot
    Aug 31 '13 at 9:46
  • $\begingroup$ in the first case, this depends on the data and used loss function, in the second one it is simple to show that $y=ax+b \iff -ax=b-y \iff x=\frac{-y}{-a}+\frac{b}{-a} \iff x = \frac{y}{a} - \frac{b}{a} \iff x = \frac{1}{a}y - \frac{b}{a}$ (assuming $a\neq 0$) $\endgroup$
    – lejlot
    Aug 31 '13 at 9:48
  • $\begingroup$ We are given 2 equations. One is X on Y and the other is Y on X with the same data-set. We are to ascertain which is which. It is simple linear regression. $\endgroup$ Aug 31 '13 at 9:49
  • $\begingroup$ There is no such thing as $X$ or $Y$, you can name your axis whatever you want, even "elephant" and "big bang". You need to put some definition what do you consider as $X$ and $Y$. from above equation you can see, that absolute value of the slope of one of this lines will be not greater then $1$, but it does not mean, that this has to be $X$ on $Y$ or vice versa $\endgroup$
    – lejlot
    Aug 31 '13 at 9:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.