Which kind of test is most suitable for comparing correlations of 3 methods?

Question

I have a R data frame like this:

structure(list(Mash_pear = c(0.328239947270445, 0.752207607551684, 
0.812118104861163, 0.640824971449627, 0.615568052052443, 0.546635339103089, 
0.557460706464288, 0.650480192893698, 0.418044504894929, 0.52962586938499
), tRap_pear = c(0.0350096175177328, 0.234255507711743, 0.23714999195134, 
0.185536020521134, 0.191585098617356, 0.201402054387186, 0.220911538536031, 
0.216072802572045, 0.132247101763063, 0.172753098431029), Beeml_pear = c(0.179209909971615, 
0.79129167285928, 0.856908302056589, 0.729078080521886, 0.709346164378725, 
0.669599784720647, 0.585348196746785, 0.639355942917055, 0.544909349368496, 
0.794652394149651), Mash_pear50 = c(0.192474082559755, 0.679726904159742, 
0.778564545349054, 0.573745352397321, 0.56633658385284, 0.472559997318901, 
0.462635414367878, 0.562128414492567, 0.354624921832056, 0.64532681437697
), labels = c("Aft1", "Alx3", "Alx4", "Arid3a", "Arid3a", "Arid3a", 
"Arid3a", "Arid5a", "Arid5a", "Aro80"), fam = c("AFT", "Homeo", 
"Homeo", "BRIGHT", "BRIGHT", "BRIGHT", "BRIGHT", "BRIGHT", "BRIGHT", 
"Zn2Cys6"), pwmlength = c("21", "17", "17", "17", "17", "17", 
"17", "14", "14", "21")), .Names = c("Mash_pear", "tRap_pear", 
"Beeml_pear", "Mash_pear50", "labels", "fam", "pwmlength"), row.names = c("Aft1", 
"Alx3_3418.2", "Alx4_1744.1", "Arid3a_3875.1_v1_primary", "Arid3a_3875.1_v2_primary", 
"Arid3a_3875.2_v1_primary", "Arid3a_3875.2_v2_primary", "Arid5a_3770.2_v1_primary", 
"Arid5a_3770.2_v2_primary", "Aro80"), class = "data.frame")

The first 4 columns are my correlations which i want to test for significant differences. These 4 columns are methods to estimate transcription factor binding to the DNA. Now i want to know which method performs best? I tried a paired t-test and unpaired t-test which seems the most suitable to me. Now i am wondering on how to interpret the test and are there other ways to test which method is better.

Data.frame for readability:

                         Mash_pear  tRap_pear Beeml_pear Mash_pear50 labels     fam pwmlength
Aft1                     0.3282399 0.03500962  0.1792099   0.1924741   Aft1     AFT        21
Alx3_3418.2              0.7522076 0.23425551  0.7912917   0.6797269   Alx3   Homeo        17
Alx4_1744.1              0.8121181 0.23714999  0.8569083   0.7785645   Alx4   Homeo        17
Arid3a_3875.1_v1_primary 0.6408250 0.18553602  0.7290781   0.5737454 Arid3a  BRIGHT        17
Arid3a_3875.1_v2_primary 0.6155681 0.19158510  0.7093462   0.5663366 Arid3a  BRIGHT        17
Arid3a_3875.2_v1_primary 0.5466353 0.20140205  0.6695998   0.4725600 Arid3a  BRIGHT        17
Arid3a_3875.2_v2_primary 0.5574607 0.22091154  0.5853482   0.4626354 Arid3a  BRIGHT        17
Arid5a_3770.2_v1_primary 0.6504802 0.21607280  0.6393559   0.5621284 Arid5a  BRIGHT        14
Arid5a_3770.2_v2_primary 0.4180445 0.13224710  0.5449093   0.3546249 Arid5a  BRIGHT        14
Aro80                    0.5296259 0.17275310  0.7946524   0.6453268  Aro80 Zn2Cys6        21 

Answer 1

I'm not entirely sure how you have created the data and what the columns mean. However, here is my general take:

1. Make sure that what you are comparing are independent observations. The assumption of independence is thought to be more important than, for example, the assumption of normality. If your observations are not independent (as might be the case here), you should refrain from using a statistical testing procedure that assumes independence.

2. If your observations are independent, then use a non-parametric test, as a correlation coefficient clearly has not a normal distribution (for example because -1 < r < 1).

3. Use a randomization procedure to obtain a H0 distribution; this might save you even if your observations are not independent. The details would depend on how these correlations were derived.