What is the distinction between "relationships" and "differences" (particularly in the context of group differences on a numeric attribute)?

Question

What is the distinction between a relationship and a difference in the context of statistics - if you have a nominal variable and an interval variable, when should you use a correlation to measure the relationship, rather than a t-test or an ANOVA?

I'm not after the usual 'what is your question about - relationships or differences' (although I obviously appreciate the importance), but rather what is it about the data or the constructs that I should know to make this decision.

If for example you have males and females and their intelligence scores. Is there a statistical/conceptual reason why a t-test is better here than a point-biserial correlation?

This is probably going to be an obvious question. To be honest though, I don't feel like I've ever really understood the distinction, so perhaps some of you can articulate it for me?

Answer 1

Association (or "correlation") and difference are two sides of the same coin. Difference between groups by some quantitative characteristic can be reasoned as the association between variables "group" and "characteristic". Eta coefficient, also called correlation ratio, is the proper association measure between a nominal variable and a scale variable, and is therefore the other side of the coin for ANOVA or t-test. In fact, Eta is used as the effect-size measure (i.e. standardized measure of the size of the difference) in ANOVA. Association measures which vary from 0 to 1 or -1 to 1 are often used as effect size measures. Point-biserial correlation is equal to Eta when there are 2 groups. Point-biserial correlation is just Pearson r.