# Validity of reporting correlation with N = 4

Experimental Design

4 Animals repeated a binary choice task with 60 trials over 15 days. For each day, we calculated the proportion for which the animal chose the given 'correct' stimulus. In addition to this choice data, we have a single continuous behavioral measure for each animal (4 data points). We wish to relate this behavioral measure with the animals' overall proportion of 'correct' choices. You can think of this variable as a characteristic of each animal, such as its sex or gene variant, etc.

Initial Analysis

We initially calculated the mean proportion for each animal across all days and calculated the Pearson Correlation of the mean proportion with the behavioral measure (-.99 [-.999,-.3160], 95% CI); however, a reviewer commented on how this was meaningless as we have such a small N.

Question: What other method(s) should we utilize to demonstrate this relationship?

Data

As requested by whuber, the data is as follows:

 means   behavior
0.672         13
0.637         21
0.596         23
0.513         39


As per my paranoia, the numbers are changed quite minimally. This should not alter any of the interpretations.

• More information is needed if you want good answers. E.g., suppose the total numbers of correct choices made were $0,1,3,4$ for animals with behavior scores of $1,.9,.15,0$ respectively (which is consistent with your description: $\rho=-0.998$, CI=$[-0.9996,-0.41]$). These counts have Binomial distributions; assuming the observations reflect the true chances, there's less than a $1$% chance of a positive correlation. But if the numbers were $30,31,33,34$--equally consistent with your description--there is now a $21$% chance of a positive correlation.
– whuber
Commented Jun 10, 2013 at 14:28
• That is very interesting to me. Can you point me to where I can find more information on the relation between binomial count and correlation? The counts of correct answers for each animal range from about 300 to 700, but I don't recall ever learning anything about the magnitude of count's effect on correlation. Commented Jun 10, 2013 at 15:35
• With that range you're probably ok, but it would be worth looking more closely. (The idea is that the variance of the counts is approximately proportional to the counts themselves, so when the spacing between the counts is of the same order of magnitude as their square roots, you cannot reliably order the counts.) Since your data amount to just four pairs of numbers, perhaps you could post them? There could also be concerns about the "behavioral measure": how is it assessed and how accurate and reliable is it?
– whuber
Commented Jun 10, 2013 at 16:14
• Ah, that makes sense. I don't have the data on hand, but I agree with your point about the behavioral measure; the reviewers did mention some concerns. Thanks for the assistance--when I get a hold of the data I'll follow up. Commented Jun 10, 2013 at 16:38