There is a well-answered question here.

But unfortunately, I don't even understand how the first equation in the answer is derived. Could someone help explain that?

$$\text{Beta:} \quad \beta_{x_1} = \frac{r_{yx_1} - r_{yx_2}r_{x_1x_2} }{1-r_{x_1x_2}^2}$$

What is $r$ here?

  • 3
    $\begingroup$ $r_{ab}$ is the correlation between $a$ and $b$. $\endgroup$ – Glen_b Jun 8 '16 at 8:44
  • $\begingroup$ Thanks @Glen_b but I am still not sure I understand this equation. I think this $\beta$ is the same as $(X_1^TX_1)^{-1}X_1^TY$, how is this related with the right-hand-side? $\endgroup$ – Haohan Wang Jun 8 '16 at 18:34
  • $\begingroup$ Thanks. @Glen_b I found it here $\endgroup$ – Haohan Wang Jun 8 '16 at 19:00
  • 2
    $\begingroup$ how is this related with the right-hand-side? First, consider the formula (the 1st one here which is homologic with the right-hand one but is for $b$, not for $\beta$. Both this and that are found in nearly any book on linear regression. Second, considering X having just 2 columns, "unwrap" algebraically and in scalar (in place of matrix) notation the expression $(X^TX)^{-1}X^TY$; and I'm sure you will arrive exactly at the formula for $b$. $\endgroup$ – ttnphns Jun 8 '16 at 20:51

As the comment points out, $r_{ab}$ is the correlation between $a$ and $b$.

This equation: $$\text{Beta:} \quad \beta_{x_1} = \frac{r_{yx_1} - r_{yx_2}r_{x_1x_2} }{1-r_{x_1x_2}^2}$$ can be achieved by solving the following equation:

$$ \beta_{x_1} + r_{x_1x_2}\beta_{x_2} = r_{x_1y} \\ r_{x_1x_2}\beta_{x_1} + \beta_{x_2} = r_{x_2y} $$

where $x_1$ and $x_2$ are two predictors and $y$ is the dependent variable. Solving this above equation set by basic linear algebra will lead to the first equation.

Details can be found in Chapter 5 Multiple correlation and multiple regression of An introduction to psychometric theory with applications in R by William Revelle.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.