Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
There is a well-answered question here.
But unfortunately, I don't even understand how the first equation in the answer is derived. Could someone help explain that?
$$\text{Beta:} \quad \beta_{x_1} = \frac{r_{yx_1} - r_{yx_2}r_{x_1x_2} }{1-r_{x_1x_2}^2}$$
What is $r$ here?
As the comment points out, $r_{ab}$ is the correlation between $a$ and $b$.
This equation: $$\text{Beta:} \quad \beta_{x_1} = \frac{r_{yx_1} - r_{yx_2}r_{x_1x_2} }{1-r_{x_1x_2}^2}$$ can be achieved by solving the following equation:
$$ \beta_{x_1} + r_{x_1x_2}\beta_{x_2} = r_{x_1y} \\ r_{x_1x_2}\beta_{x_1} + \beta_{x_2} = r_{x_2y} $$
where $x_1$ and $x_2$ are two predictors and $y$ is the dependent variable. Solving this above equation set by basic linear algebra will lead to the first equation.
Details can be found in Chapter 5 Multiple correlation and multiple regression of An introduction to psychometric theory with applications in R by William Revelle.
how is this related with the right-hand-side?First, consider the formula (the 1st one here which is homologic with the right-hand one but is for $b$, not for $\beta$. Both this and that are found in nearly any book on linear regression. Second, considering X having just 2 columns, "unwrap" algebraically and in scalar (in place of matrix) notation the expression $(X^TX)^{-1}X^TY$; and I'm sure you will arrive exactly at the formula for $b$. $\endgroup$ – ttnphns Jun 8 '16 at 20:51