# Tests for comparing correlation values. The winner is?

The question is relatively simple, but trivial.

I have data divided by one fixed effect (Emotion, with 2 conditions: A and B), and I have a covariate, for each of my 20 subjects.

My aim is to

1) evaluate correlation between data and covariate;

2) know if there is a significantly bigger correlation between data in A condition and A covariate, than between data in B condition and B covariate (or viceversa). That means, analitically, to compare regression slopes.

I evaluated five different strategies:

a) use of Cohens'q. That gives no probability of a significance (no p-value), but an 'effect size' (small/medium/large) based on the difference between r values (transformed in z values using Fisher's procedure).

b) use of Fisher's method. That takes in account sample size, other than transformed r values.

c) use of ANCOVA. In particular, an analysis of covariance that doesn't force the slope to be the same. MatLab's 'aoctool' can do that work. To be clear:

 y = (a1 + a2) + (b1 + b2)*X + e


In this case, not only sample sizes are taken in account, but also variability in each group.

d) Bootstrap procedure, followed by an effect size measure (Cohens'd).

e) Linear Mixed Effect model. It is usually indicated as a method which provides better estimates by using both group-level and population-level information (Thanks to @mzunhammer).

Cohens'q is just about correlations. Instead, Fishers' method takes also in account the sample size, and ANCOVA/LME methods also account for variability; but the latter two methods, with a small sample size, will tend to reject a difference between slopes. Actually, in neuroimaging studies, it is common to have 15-25 subjects. Therefore it appears to be useful to use Cohens'q, to have an estimate of the difference between correlations.

# QUESTIONS

What would you recommend to solve the point 2)?

Does the bootstrapapproach seem reasonable?

What you want is an General Linear Model of the form

y = b0 + emob1 + rtb2 + emortb3

where

• b0: is the intercept
• b1: is the main effect of emotion (positive/negative)
• b2: is the main effect of reaction time (regardless of condition)
• b3: is the interaction effect of emotion and reaction time (effect of reaction time depending on condition)

If you dummy-code emotion as neutral=0 and negative=1 the main effect rt will represent the association between y and reaction time at neutral conditions and the interaction term will represent the additional change in association due to negative conditions.

Make sure to center or standardize (z-Score) your reaction-times beforehand. If you do so, b1 will represent the effect of emotion at the overall mean reaction time (for a subject of average performance). If you forget to do so b1 will represent the effect of emotion for a reaction time of 0 (and thus nonsense)...

The big question is whether your experimental design was within-subject (all 20 subjects had both emotional conditions) or between-subject (some only negative, some only positive condition).

Between subject: do the ancova (emo+rt+emo*rt).

Within subject: Things will get a little more involved, because there is no repeated-measures ANCOVA for changing continuous covariates. In this case you have to use a mixed model (aka random effects model). Here, the fixed part of the model should be defined exactly like for the ANCOVA, but you should add the following random effects: – by-subject intercept (each subject gets its own intercept) – eventually also by-subject random effect for emotion (each subject gets its own effect estimate of emotion... however this will only be estimable if you have multiple repetitions/trials for each subject and condition)

• Thanks. Experimental design is a full within-subject. So, you are suggesting a GLM with dummy 0/1 codes for b1, and standardized reaction times for b3. Although I didn't get what did you suggest for b2, this doesn't seem to be an ANCOVA, and neither a mixed effects model. Actually, I'm quite sure that MatLab offers the possibility to run a repeated measures ANCOVA, using 'continuous' for the covariate and 'random' for the repeated measures label. Of course, one should specify in the model that interaction is required. But, still, I'm not sure of what is your primary suggestion... – smndpln Nov 28 '16 at 23:16
• Ok, so it's within-subject design and MATLAB. – mzunhammer Nov 29 '16 at 22:38
• 1.) "Analysis of covariance (ANCOVA) is a general linear model[...]" en.wikipedia.org/wiki/Analysis_of_covariance – mzunhammer Nov 29 '16 at 22:54
• 2.) Repeated-measures ANCOVAs cannot handle time-changing co-variates, see: link MATLABs fitrm is no exception here. How would you enter one column with reaction-times for neutral and positive conditions at the same time in the required wide format? If you know such a function in MATLAB please name it, I would be very interested. – mzunhammer Nov 29 '16 at 22:54
• 3.) Create a table with variables "subjectID", "outcome", "emotion", and "rt" in "long" format (repeated measures below each other). Make sure to dummy-code emotion and to standardize rt. Then try lme = fitlme(tbl,'outcome~emotion+rt+emotion*rt+(1|subjectID), 'CheckHessian', true) see: de.mathworks.com/help/stats/fitlme.html. If you have more than one repetition per subject and condition try: lme = fitlme(tbl,'outcome~emotion+rt+emotion*rt+(1+emotion|subjectID), 'CheckHessian', true) – mzunhammer Nov 29 '16 at 22:57