3
$\begingroup$

I have performed 61 Pearson correlation analyses between Variable A (buying ice cream) and Variable B (buying yoghurt) in my dataset (n=550). Each correlation is based on a different subset of the data (e.g., separate correlations for males and females; separate correlations for different age groups).

Fortunately (or unfortunately!), for all 61 correlations, my p value is less than 0.0007431 (which is the correction of 0.05 using Sidax).

I interpret this to mean that all the findings are significant, which in my cases is that there is a relationship between Variable A (people who buy ice cream) and Variable B (people who buy yoghurt).

Question

  • Is it normal to get all significant correlations when analysing the same correlation across a large number of subsets of data?
  • Is it an acceptable results (i.e., "is not questionable" etc.)?

I can assure that my procedures for data collection and analysis are robust.

$\endgroup$
6
  • $\begingroup$ 61 different correlation analyses between the same variables in the same dataset? How did you come to this? $\endgroup$
    – Nick Sabbe
    Sep 15, 2011 at 11:28
  • $\begingroup$ For example, by gender (2 correlations), marital status (3 correlations), age groups (4 correlations), language (4 correlations) and the list goes on! $\endgroup$ Sep 15, 2011 at 11:41
  • 2
    $\begingroup$ @Adhesh, how exactly do you calculate the correlation between gender and language? All of the examples you gave in the above comment (with perhaps the exception of age groups) have no numerical representation with which one could calculate a Pearson correlation coefficient. $\endgroup$
    – Andy W
    Sep 15, 2011 at 14:10
  • $\begingroup$ @andy. Sorry if I have confused you. My correlation is between "buying ice-cream, as measured in litres" and "buying yoghurt as measured in litres". I am then comparing them by gender, age etc. of the shoppers. $\endgroup$ Sep 15, 2011 at 20:32
  • 2
    $\begingroup$ @Adhesh, no need to apologize, it is necessary information to answer your question though. Long story short, if none of the characteristics of the shoppers themselves are correlated with both buying yogurt and buying ice-cream, then the correlation will not change when examining different sub-groups. See this answer I gave for another question. $\endgroup$
    – Andy W
    Sep 15, 2011 at 20:45

1 Answer 1

1
$\begingroup$

(I know it's 12 years later, but I'm just going through some unanswered questions).

Is it normal to get all significant correlations when analysing the same correlation across a large number of subsets of data?

Well, here, it seems like you have just randomly subset the data by every variable you can think of. There doesn't seem to be any strong reason to think the correlations would differ by demographic group.

Is it an acceptable results (i.e., "is not questionable" etc.)?

Every result can be questioned. That's how science works. One question to ask here is why you are looking at all these correlations. Another question is why you are judging these by "signficant" or "not significant" which isn't a very useful distinction. You could better look at whether the correlations vary much.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.