2
$\begingroup$

How can I test if two averages of Pearson's correlation coefficients are significantly different?

For instance, I have data pertaining to participants' estimates of certain percentages (e.g. how many percent of US-citizens are taller than 6 feet) in a number of different conditions.

For instance, I want to know whether the average degree of correlation between participants is significantly different across conditions. I have data from 20 participants who each answer the same 10 questions in two different conditions. Within each condition it is straightforward how to estimate the average between participant correlation; just calculate a Pearson's for the answers of each pair of participants and then divide the sum of these correlations with the number of comparisons. But how do I determine whether or not these averages are significantly distinct?

At first I thought that I could use a test for the difference between non-independent r's (p.281 in Howell's "Statistical Methods for Psychology"). But occasionally I get results where the equation requires me to take the square root of a negative number which I take as evidence that using the equation is inappropriate. So I'm at a loss of how to do this.

EDIT: Here is some background information:

I'm interested in the relation between the average validity (understood as the correlation between a participant's answers and the correct answers) of participants in a group and the validity of the mean answer for that group.

The reason I use Pearson is that there is an equation (found in e.g. Hogarth 'A Note on Aggregating opinions' in Organizational Behavior and Human Performance 21, 1978) that states the validity of the mean answer in a group as a function of the average degree of correlation between group members and group members' average validity.

Improve this question
$\endgroup$
7
  • 2
    $\begingroup$ This seems the wrong way to go. As the same correlation could reflect different linear relations, and vice versa, you would be moving yet further from the data, and correlations really don't seem a natural measure to average. What is the scientific question underlying all this? It can probably be rephrased in terms of a generating model. $\endgroup$
    – Nick Cox
    Commented Mar 3, 2014 at 13:06
  • $\begingroup$ I edited the question with some background information. $\endgroup$
    – ProfessorHelloKittyNr1
    Commented Mar 3, 2014 at 15:02
  • $\begingroup$ What is the problem with averaging correlations? And what is a generating model? $\endgroup$
    – ProfessorHelloKittyNr1
    Commented Mar 3, 2014 at 15:10
  • $\begingroup$ Good questions but difficult to answer informatively as well as concisely. A generating model reflects whatever data generating process you are imagining. A correlation can make sense as a summary for a particular dataset with just two variables; thinking that there is a Platonic average underlying correlation is a harder stretch. An alternative is to suggest that you have a multivariate problem, which would be ill-served by pretending there's a single underlying correlation; that's a hypothesis to be tested, not an assumption to be adopted. $\endgroup$
    – Nick Cox
    Commented Mar 3, 2014 at 15:21
  • $\begingroup$ Useful discussion at stats.stackexchange.com/questions/8019/… $\endgroup$
    – Nick Cox
    Commented Mar 3, 2014 at 15:41

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.