5
$\begingroup$

I am investigating the effect of a drug on EEG and cognition in epilepsy patients. We have done EEG and neuropsychological (NP) tests twice, before and after medical treatment. Some of parameters in EEG and NP tests were significantly altered after drug treatment (done by Wilcoxon rank sum test, as the distribution was not normal).

Now, I would like to calculate correlation between change in EEG parameters and change in NP tests. There are 7 parameters in EEG parameters and 27 in NP tests. I would like to run correlation analysis on every pair and see if any pair is significantly correlated.

My questions are:

  1. Should I use nonparametric correlation, say Spearman, to calculate $\rho$? (as values of EEG and NP paramaters are not normal)

  2. Since there are 7 parameters in EEG and 27 parameters in NP tests, there should be 7*27 number of testing, which requires correction of p value for multiple comparisons. Which method should I use?

I am looking for not-so-conservative method. I have seen from another posting that I can use permutation test for multiple comparisons. Could someone explain how I can do this?

Improve this question
$\endgroup$
1
  • $\begingroup$ About the permutation approach, see @whuber's answer to this related question, Look and you shall find (a correlation), which describes the basic idea of breaking the structure or link between your two series of measurements. $\endgroup$
    – chl
    Mar 15, 2011 at 9:55

2 Answers 2

Reset to default
4
$\begingroup$

In my opinion you should test if your variables are distributed normally and chose a suitable test accordingly.

Concerning the correction for alpha inflation: What you are doing is data mining. You have experimental data and now you are digging in it to find ... anything. Do that. But also know that anything you might find is just an observation and as such not reliable. Perform that exploratory thing, pick some promising pairs of variables and conduct new experiments to test these pairs for correlation.

Improve this answer
$\endgroup$
1
  • $\begingroup$ This seems a little harsh. It sounds like the search for correlations was planned at the outset and the OP is aware of the multiple comparisons problem. That already puts them on a better experimental, analytical, and conceptual footing than the vast majority of publications in this field. $\endgroup$
    – whuber ♦
    Mar 15, 2011 at 15:25
3
$\begingroup$

You want to perform a canonical correlation analysis. This will provide information about correlations among linear combinations of your sets of parameters, potentially uncovering stronger information of the type you seek. The Wikipedia article explains the theory, provides the equations, and presents the appropriate hypothesis test. It requires solving a 7 by 7 eigenvalue problem determined by the correlation and cross-correlation matrices of the variables, which is fast and straightforward. The R package CCorA will do it, for instance. (I haven't tested this package.)

It might help first to re-express your original variables so they are each approximately symmetrically distributed. There are many ways to do this using, for example, Box-Cox transformations.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Dear whuber, thanks for your answer. I will try as you suggested. Meanwhile, I also read about your post with permutation testing, and it seems a do-able way for my application as well. I do have one question tho. Suppose I ran 10000 iterations with permuted variables, and have almost normal distribution. How can I calculate the p value of my "original" correlation coefficient (not permuted, experimantal data)? $\endgroup$
    – user3720
    Mar 16, 2011 at 2:16
  • 1
    $\begingroup$ @Ryan (If you want to address @whuber directly, don't forget to prefix its username with @ so that he can be notified.) I don't really like to intervene like this, but what is your "original" correlation about? I mean, a perm. test will inform you about the prob. of observing such a stat. ($r$ in your case) at least as extreme as the one you observed in your data (and with 10,000 draws, this is largely enough to conclude at a 5% level); then reporting your $p$-value is just a matter of computing the No. times you observed $r>r_\text{obs}$, where $r_\text{obs}$ = your empirical corr. $\endgroup$
    – chl
    Mar 16, 2011 at 20:41
  • $\begingroup$ @chl, thanks for your answer. Although, what do you mean by my "original" correlation? Also, you said that reporting a p-value is a matter of computing the # of times I observe r>robs. For example let's say my observed correlation value is 0.7, and in 10,000 draws, only 10 of them were larger than 0.7. Then, my p-value is 10/10,000=0.001? $\endgroup$
    – user3720
    Mar 17, 2011 at 6:31
  • $\begingroup$ @Ryan Let's say you estimate a correlation of 0.36 on your sample -- this is what I call the observed or original correlation. Now you permute the rows of one your block (this would break the link between the two blocks) and compute your cross-correlations (between each variable of each block), iterate say 10,000 times, and then yes you can compute the empirical p-value as you described. The same strategy could be used to assess the significance of the first canonical correlation in a CCA, BTW. $\endgroup$
    – chl
    Mar 17, 2011 at 8:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy