Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Since standardized betas are correlation coefficients in bivariate regression, is it the case that standardized betas in multiple regression are partial correlations?

share|improve this question
1  
Short answer: No. – Glen_b Feb 17 '13 at 2:51
    
Because the value of a beta can be anything (including of absolute value greater than $1$), it cannot generally be interpreted as a partial correlation. – whuber Feb 17 '13 at 3:00

Longer answer.

If I have this right --

Partial correlation:

$$ r_{y1.2} = \frac{r_{y1}-r_{y2}r_{12}}{\sqrt{(1-r^2_{y2})(1-r^2_{12})}} $$

equivalent standardized beta:

$$ \beta_1 = \frac{r_{y1}-r_{y2}r_{12}}{(1-r^2_{12})} $$

As you see, the denominator is different.

Their relative size depends on whether $\sqrt{(1-r^2_{y2})}$ or $\sqrt{(1-r^2_{12})}$ is smaller.

share|improve this answer
1  
+1. But not least fact is that the numerator is the same. This implies that both coefficients are just different ways to standardize the raw regression coefficient b. – ttnphns Feb 17 '13 at 8:05
    
I assume the right way to read "$\beta_1$ will almost always be smaller than $r_{y1.2}$" is "smaller in magnitude". But unless I'm getting confused here, the partial correlations always satisfy $-1 \leq r \leq 1$, just like usual ones. Yet Gottfried Helm's answer to this question shows that the partial betas could exceed one in magnitude in a multiple regression, & Jeremy Miles explicitly constructs such an example on another thread about this issue, making use of multicollinearity. This suggests beta can exceed the partial $r$ in magnitude – Silverfish Nov 25 '15 at 12:33
    
I'm not sure, but I think there is no square root in the denominator of the standardized beta. This would explain how beta would exceed the partial r in magnitude. – Leo Azevedo Jan 4 at 17:03
    
@Leo You're right, that's a mistake on my part. I've fixed the formula; I've now also adjusted the comments that resulted from me getting the formula wrong. – Glen_b Jan 5 at 3:13
    
@Glen_b would you have a reference for the derivation of the partial correlation and standardized beta in the general multivariate case? I want to formalize the relationship between partial correlation, multiple regression coefficients and conditional mutual information for jointly gaussian variables, but most references point me in the direction of software, not math. Thx! – Leo Azevedo Jan 7 at 14:01

I've in another question the following covariance matrix C for the three variables X,Y,Z given:
$$ \text{ C =} \small \left[ \begin{array} {rrr} 1&-0.286122&-0.448535\\ -0.286122&1&0.928251\\ -0.448535&0.928251&1 \end{array} \right] $$ From its cholesky-decomposition L $$ \text{ L =} \small \left[ \begin{array} {rrr} X\\Y\\Z \end{array} \right] = \left[ \begin{array} {rrr} 1&.&.\\ -0.286122&0.958193&.\\ -0.448535&0.834816&0.319215 \end{array} \right] $$ we can directly retrieve the partial correlation between Y,Z wrt. X as $ \small corr(Y,Z)_{\cdot X} = 0.958193 \cdot 0.834816 $ Now if we have the variables ordered such that the dependent variable is Z then the betas are computed by inverting the square-submatrix of the range in L which is populated by the independent variables X,Y: $$ L_{X,Y} = \small \left[ \begin{array} {rrr} 1&.\\ -0.286122&0.958193 \end{array} \right] $$ and its inverse, which is inserted into a 3x3 identity-matrix to form the matrix $M$: $$ M = \small \left[ \begin{array} {rrr} 1&.&.\\ 0.298605&1.043631&.\\ .&.&1 \end{array} \right] $$ Then the betas occur by the matrix-multiplication $ \beta = L \cdot M $

$$ \beta =\small \left[ \begin{array} {rrr} X\\Y\\Z \end{array} \right] = \small \left[ \begin{array} {rrr} 1&.&.\\ .&1&.\\ -0.199254&0.871240&0.319215 \end{array} \right] $$

which indicates, that the $\beta_X$ contribution for $Z$ is $\small \beta_X=-0.199254$ and the $\beta_Y$ contribution for $Z$ is $\small \beta_Y=0.871240$ . The unexplained variance in Z is the bottom-right entry squared: $\small resid^2= (0.319215)^2$
We see in $M$ that -being an inverse of a partial cholesky-matrix- it can contain values bigger than $1$ - and as well the Beta-matrix can then have entries bigger than 1.

So - to come back to your question- the partial correlation between $Y$ and $Z$ were the product of the entries in the second column of the L-matrix. The $ \small \beta_Y$ however is the product of the entry in the second column of the Z-row with the inverse of that in the Y-row and the relation between the concepts of partial correlation and $\small \beta$ can be described by this observation.

Additional comment: I find it a nice feature, that we get by this also the compositions of $X$ and $Y$ in terms of $X$ and $Y$ - which of course are trivially 1. It is also obvious, how we would proceed, if we had a second dependent variable, say $W$, and even that scheme can smoothly be extended to compute/show the coefficients of the generalization to the canonical correlation - but that's another story....

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.