Following on from my earlier posts, as far as I can understand, if I have three correlation coefficients, I will have to test them in pairs to see if there is a significant difference among them.

This means that I would have to use Fishers transformation to work out the z score of r and then the p value of z (which the recommended calculators in the earlier posts do, thankfully) and then ascertain if the p value is higher or lower than my alpha value (0.05) for each pair.

i.e. If 21 to 30 years olds is Age Group 1, 31 to 40 years is Age Group 2, and 41 to 50 years is Age Group 2, my comparison of the correlations between their shopping habits and weight loss would be:

  • Group 1 vs. Group 2
  • Group 1 vs. Group 3
  • Group 2 vs. Group 3

Instead of doing three separate calculations, is there a way to do all these calculations in a single step?

  • 1
    $\begingroup$ Could you please be a bit more detailed? As in -- what is your response, your explanatory variables, and what correlations are you interested in? You might not Fisher's transform for testing correlation, a simple t-test may be sufficient. $\endgroup$
    – suncoolsu
    Sep 3 '11 at 17:55
  • $\begingroup$ @suncoolsu I am testing the correlation between shopping habit and weight gain for these three groups. My results are as follows: Group 1: r=.8978, n=105; Group 2: r=.5678, n=95; and Group 3: r=.7865, n=120. $\endgroup$ Sep 4 '11 at 1:05
  • $\begingroup$ I think your data passes the IOTT. That's the interocular trauma test - it hits you between the eyes. If correlations of .9, .6 and .8 aren't different from each other, what is? But if you're really interested $\endgroup$
    – Peter Flom
    Sep 4 '11 at 11:32

Your question is a perfect example of regression models with quantitative and qualitative predictors. Specifically, the three age groups -- $1,2, \& \,3$ -- are the qualitative variables and the quantitative variables are shopping habits and weight loss (I am guessing this because you are calculating correlations).

I must stress that this is much better way of modeling than calculating separate group-wise correlations because you have more data to model, hence your error estimates (p-values, etc) will be more reliable. A more technical reason is the resulting higher degrees of freedom in the t-test statistic for testing the significance of the regression coefficients.

Operating by the rule that $c$ qualitative predictors can be handled by $c-1$ indicator variables, only two indicator variables, $X_1, X_2$, are needed here that are defined as follows:

$$ X_1 = 1 \text{ if person belongs to group 1}; 0 \text{ otherwise} . $$ $$ X_2 = 1 \text{ if person belongs to group 2}; 0 \text{ otherwise}. $$

This implies that group $3$ is represented by $X_1=0, X_2=0$; represent your response -- shopping habit as $Y$ and the quantitative explanatory variable weight loss as $W$. You are now fit this linear model

$$ E[Y]=\beta_0 + \beta_1X_1 + \beta_2X_2 + \beta_3W. $$ The obvious question is does it matter if we change $W$ and $Y$ (because I randomly chose shopping habits as the response variable). The answer is, yes -- the estimates of the regression coefficients will change, but the test for "association" between conditioned on groups (here t-test, but it is same as testing for correlation for a single predictor variable) won't change. Specficially,

$$ E[Y]= \beta_0 + \beta_3W \text{ -- for third group}, $$ $$ E[Y]= (\beta_0 + \beta_2)+\beta_3W \text{ -- for second group}, $$ $$ E[Y]= (\beta_0 + \beta_1)+\beta_3W \text{ -- for first group}, $$ This is equivalent to having 3 separate lines, depending on the groups, if you plot $Y$ vs $W$. This is a good way to visualize what you are testing for makes sense (basically a form of EDA and model checking, but you need to distinguish between grouped observations properly). Three parallel lines indicate no interaction between the three groups and $W$, and a lot of interaction implies these lines will be intersecting each other.

How do the tests that you ask. Basically, once you fit the model and have the estimates, you need to test some contrasts. Specifically for your comparisons:

$$ \text{Group 2 vs Group 3: } \beta_2 + \beta_0 - \beta_0 = 0, $$ $$ \text{Group 1 vs Group 3: } \beta_1 + \beta_0 - \beta_0 = 0, $$ $$ \text{Group 2 vs Group 1: } \beta_2 + \beta_0 - (\beta_0+\beta_1) = 0. $$

  • $\begingroup$ Testing for the equivalence of slopes is different than testing for the equivalence of correlations. See, for example: jessicagrahn.com/uploads/6/0/8/5/6085172/comparecorrcoeff.doc $\endgroup$
    – Wolfgang
    Sep 4 '11 at 22:32
  • $\begingroup$ I agree, but for a single predictor variable, they should be identical because of this relationship $t^* = \frac{\rho\sqrt{n-2}}{\sqrt{1-\rho^2}} \sim t_{n-2}$. $\endgroup$
    – suncoolsu
    Sep 4 '11 at 22:57
  • $\begingroup$ Also, your document talks about comparing different populations, which is not the case of single predictor. $\endgroup$
    – suncoolsu
    Sep 4 '11 at 23:00
  • $\begingroup$ The point is that $H_0: \beta_1 = \beta_2 = \beta_3$ could be true, while $H_0: \rho_1 = \rho_2 = \rho_3$ could be false (and vice-versa). The correlation between X and Y depends not only on $\beta$, but also the variance in X and the variance in the errors. If the variance in X and/or the errors differs across the 3 groups, then you are testing different hypotheses. $\endgroup$
    – Wolfgang
    Sep 5 '11 at 12:42
  • $\begingroup$ Yes you are right (as I said before), but my response assumes that the OP was interested in determining the relationship between wt.loss and shopping habits based on groups (not necessarily correlation). I guess I was wrong because the OP accepted the other answer. Nonetheless, this answer serves as an useful alternative (I hope). $\endgroup$
    – suncoolsu
    Sep 5 '11 at 14:09

Pairwise testing in this situation is not (yet) justified by the data description. You should be using multi-variable regression methods. An R call might be:

lm( weight_end ~ shop_habit + age_grp + weight_begin)

Constructing 3 categories is not the best method of controlling for age (or analyzing its contribution if that is the primary question) since categorization can distort continuous relationships, and spline terms remove the need to pick arbitrary split-points. Once there os sufficient evidence of an association of weight change after a proper analysis, then there will be ad-hoc test options that can deployed.

(I did agree with most of what @whuber expressed in a comment, and I generally find his commentary authoritative, but do not understand his stance regarding regression approaches.)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.