6
$\begingroup$

I am running Pearson's correlation on an overall sample of 400 respondents.

When I isolate male and female responses, my sample becomes 220 male responses and 180 female responses.

If I further isolate male and female responses by (say) age groups, some sample sizes become as low as 35 responses (for example, for females over the age of 65).

My question: How good are these sample sizes for correlation analysis? (I am looking at the relationship between income levels and overseas travel.)

(I think this has something to do with margin of error but how does this apply to inferential analysis which is based on probability. I can understand its role in descriptive statistics such as results of a political poll).

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ From my experience as a personality psychologist: I do not trust correlations with n < 80, better is a n of 100-120. In this region, correlations get stable (this is of course only a rule of thumb and certainly depends on the magnitude of the correlation). The p value is a bad guidance, as in small samples the CIs are very huge. E.g., the CI for r = .34 (n = 35) goes from .008 to .60, which is "no association" to "a strong association". Furthermore, r is rather suceptible to outliers, which is even more serious in small samples. $\endgroup$
    – Felix S
    Sep 21, 2011 at 14:54
  • 2
    $\begingroup$ @Felix Yes, one nagging concern I have had throughout this series of questions is that regardless of $n$, a single outlier can determine $r$: it need only be large enough. $\endgroup$
    – whuber ♦
    Sep 21, 2011 at 16:03
  • $\begingroup$ @whuber Thanks. I am detecting outliers using scatterplot. $\endgroup$
    – Adhesh Josh
    Sep 21, 2011 at 22:54

1 Answer 1

Reset to default
7
$\begingroup$

When it comes to sample size, bigger is better, but we often have to take what we get. With the smaller sample sizes, your estimates of the correlation are going to become extremely noisy, and comparisons between different estimates (which I expect is your primary goal in the subsets analyses) are going to be particularly noisy.

This online tutorial on standard errors (as a pdf) contains formulas for the SE of the correlation coefficient (as well as of the Fisher transformation of the correlation, which is a better scale to be measuring the SE). You'll see that the scales by approximately $1/\sqrt{n}$.

For a correlation of about 0.5, the SE with a sample size of 200 will be about 0.06; with a sample size of 50 it will be about double that.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Can't thank you enough! One of my sub-group sample size is 15 and r is 0.9208. Using the formula in the PDF, I have calculated SE to be 0.1081. Does this mean Margin of Error of 11%? $\endgroup$
    – Adhesh Josh
    Sep 21, 2011 at 22:57
  • $\begingroup$ Definitely look at a scatterplot, and note the comments above regarding outliers. For correlations near the boundaries (1 and -1), the SE is not a good measure (your estimate can't be more than 0.08 too small, but an be too large by a lot), unless you look at the Fisher-transformation scale. $\endgroup$
    – Karl
    Sep 21, 2011 at 23:32
  • $\begingroup$ @Karl would you mind re-linking the tutorial in case you still find it? Thanks $\endgroup$
    – Sos
    Dec 7, 2015 at 20:53
  • 1
    $\begingroup$ @Sosi You can find it on web archive: here. $\endgroup$
    – Karl
    Dec 8, 2015 at 4:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.