# How to measure reproducibility for repeated measures

I have run an experiment where in a series of trials, each of 10 animals was presented with each of 6 stimuli, and each stimulus was offered once to the left and once to the right nostril. Overall, each animal participated in 12 trials (6 stimuli x 2 nostrils). My point is to calculate the reliability of the data by measuring the correlation between the observations from the left vs right nostril. Previously, I used Pearson's r, but a reviewer pointed out that it is unsuitable for repeated measurements and non-normally distributed data. Could you advise me on a test I could use instead of Pearson? I would be very grateful - I tried to check by myself but with no success.

Regarding details: I measured the frequency (in numbers) and duration (in seconds) of chosen behaviors (e.g., stomping - it was about horses). So, my goal was to simply correlate the frequency of behaviors in a given category when presented on the left with the frequency of the same behaviors when presented on the right (e.g., frequency of stomping when the stimulus was presented on the left with the frequency of the same behavior when presented on the right). I wanted to do it for all behavioral categories, separately for frequency and duration.

• Welcome Aga :-) Could you add information about what you measured? Something expressed by a quantity, a yes/no reaction, or else?
– Ute
Jun 12, 2023 at 17:37
• Thank you very much for your kind question :-) Of course - I measured the frequency (in numbers) and duration (in seconds) of chosen behaviors (e.g., stomping - it was about horses). So, my goal was to simply correlate the frequency of behaviors in a given category when presented on the left with the frequency of the same behaviors when presented on the right (e.g., frequency of stomping when the stimulus was presented on the left with the frequency of the same behavior when presented on the right). I wanted to do it for all behavioral categories, separately for frequency and duration.
– Aga
Jun 13, 2023 at 8:12
• you could put these details in your question. Maybe just as an extra paragraph "details" :-)
– Ute
Jun 13, 2023 at 8:57

Spearman's correlation statistic makes no assumption about the distributions, so it will be suitable for this.

• Chrishmorris, thank you very much! Is Spearman ok to measure the correlation between repeatable data?
– Aga
Jun 13, 2023 at 15:47
• By repeatable you mean that the test for frequency is not really independent from the test for duration? I don't think that there is any way of controlling for that. I suggest that your reliability estimate should be based on whichever is lowest of the two correlations. Alternatively if the two correlations are very different, then your data conflict with the reasonable assumption that the most stimulated horses begin by stamping fast, and also continue stamping for longer. In that case it is probably time for thought. Jun 14, 2023 at 14:37
• Dear Chrishmorris, thank you once again for your kind help! Your advice sounds very reasonable. Just to clarify: by repeatable, I mean that the data used for the correlations were collected from the same horses presented with different stimuli, once on the left and once on the right side. For example, horse X could perform the stomping 10 times during the first session, 5 times during the second session, 4 times during the third session - it was a response to different stimuli (e.g., tiger urine; horse feces) but we look at it together because the stimuli were always presented on the left.
– Aga
Jun 14, 2023 at 17:51
• This way, '19' (10+5+4) would stand for the frequency of this horse in the category of stomping when the stimuli were presented on the left. Analogically, we would do the same counting for the right side. So you see, the measures we use come from the same horses but from different sessions. Do you think I can use Spearman anyway or is there something to better control for repeatability? Thank you very much for sharing your knowledge with me!
– Aga
Jun 14, 2023 at 17:55
• Spearman is fine. The fact that they came from the same horse is the reason why you expect them to come from the same distribution. If the correlation is high, then this suggests that you have a repeatable test. If it is low, then the random component of the result seems to overwhelm the effect under investigation. Jun 16, 2023 at 7:26