Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Suppose I have two variables, x and y:
say, x=[34,45,22,11,67,89] y=[14,56,23,56,89,34]
Taking Pearson's correlation between these two values gives 0.24. Is this valid and/or accurate measure of the similarity between these two variables?
If no, how to determine the number of samples required for an accurate measure of correlation?
You will get an estimate of Pearson and Spearman correlation with as few as two distinct values, meaning that your data can be shown as two distinct points on a scatter plot. The estimate of correlation will be $1$, $-1$ or $0$ and it is quite possible that such a correlation is a reasonable answer. (In this circumstance of two distinct data points, Pearson and Spearman correlation give identical answers; that is not true in general.)
In principle, we can only say how reasonable a correlation is if we have (a) a good idea of the pattern of the rest of the data (b) a good understanding of the behaviour of the system in question. In practice, naturally, (a) or (b) or both may be lacking.
So, there is no absolute minimum apart from that limit of 2. But any instinct to distrust results from a small sample is quite right. Much of, if not most, twentieth-century statistics pivots on distrust of small samples and how one should cope with the resulting uncertainty. The main principle is that uncertainty decreases smoothly(*) with sample size: there is absolutely no magic threshold that separates unreliable from reliable correlations.
With a small sample it is, for example, very likely that a few points, perhaps as few as one, dominate the calculation and we need to know whether the calculated result makes sense.
Regardless of that, it is always a good idea to plot the data to see if the correlation makes sense. The example data you give are indeed illustrative of a weak positive correlation. I would not call any data point an outlier, nor do I want to call the pattern nonlinear, although as always part of the problem is that the sample size is small. I confirm Pearson correlation of $0.24$ and add Spearman correlation of $0.23$.
Note a small clash of terminology: "number of samples" (terminology in several scientific fields, especially when the samples are physical, soil, sediment, water, blood, whatever) is for statistically-minded people "sample size". That is, in statistical terms, you have one sample with 6 observations here.
(*) The wording "smoothly" won't satisfy those with a taste or talent for mathematical rigour. Sample size clearly changes discretely. The key point is that there is no threshold.
The sample size can limit the level of significance of the test result, but the test statistic can be calculated meaningfully.
It's important to bear in mind that the correlation coefficient is, by itself, a descriptive measure. For inferential testing, the usual null hypothesis used with Pearson or Spearman correlations is $$ H_0: \rho = 0 $$ and directional and non-directional alternative hypotheses can be tested.
Using the example computation given for both Pearson and Spearman correlations by Sheskin [2007], we have 5 paired data points: (20,7), (0,0), (1,2), (12,5) and (3,3). The question is asked whether there is a significant correlation between variable 1 and variable 2.
A Pearson correlation of 0.955 is computed, but we haven't yet tested whether the relationship is significant (i.e. can we reject the null hypothesis that $\rho=0$?)
Because we only have 5 samples we have $$df = n-2$$ So 3 degrees of freedom. Using a table of critical r values we can look up the critical value of r. Again looking at Sheskin (Table A16; or for an online table), we see that the two-tailed r value at 0.05 and 0.01 significance are 0.878 and 0.959. If we employ the non-directional alternative hypothesis $$H_1: \rho \ne 0$$ we can see that the alternative hypothesis is not supported at the 0.01 significance level. This is because the sample size is very small.
In regards to the second part of your question, when $df$ becomes large the critical values at a given level of significance become very small and very small correlations can be statistically significant. This does not mean that there is a causal relationship, or even a useful relationship between the variables.
