I am dealing with a strange situation which I hope you can come up with a solution. I have 3 pared group(n=7 for each) which I tried to see if they are significantly difference and if so, in which pairs. The means for Groups (G1-3) are 33.01, 31.20, and 22.93 respectively. The variances are 129.10, 116.56, and 75.12.
To test the normality, I used Shapiro-Wilk and d’Agostino-Pearson test together with Box plot. I have 1 outlier in G2 and G3 according to QQ plot (although Outliers in ANOVA, did not find any outliers with Alpha=0.02) and both normality tests were negative for G3. The Levene’s Tests (3 types) show that the 3 groups show that p value>0.5 (actually all of them are p>0.5) which I concluded that the samples are homogeneous (I’m not sure how it would work with the outliers). Anyhow, because the assumptions for ANOVA were not met completely, I did 2 things and compared the results:
1- I ran the same tests on LOG10 of my data and the results are as follows: Means for G1-3: 1.50, 1.47, and 1.34 and variance is 0.020, 0.021, 0.018 respectively. This time, only d’Agostino-Pearson test for G3 was p0.8 (homogeneous data) and the Box and Whisker diagram showed an outlier in Log G3. So my first question is if I running one way ANOVA would be recommended? Anyhow, I did the run the one way ANOVA (assuming that the variances are close enough) and p-value> 0.05 for 3 groups. I ran post hoc Contrast test and by setting c=1 for Log G1 and c=-1 for Log G2 (combination of Log G1- Log G3) , the result is ‘slightly’ significant (p=0.0498) but other compbinations show no significance. This didn’t repeat when the p-value for Tukey HSD tests were >0.05 for all 3 combinations of Log (G1-G2, G1-G3 and G2-G3). I ran non-parametric Schaich-Hamerle and Kruskal-Wallis tests followed by Contrast KW and the results of all of them are insignificant. Logically we should assume that the groups are equal and we don’t need to run the paired tests but because the mean of G3 is lower than 2 others and that the Contrast test for ANOVA was insignificant for G1-G3, I ran two paired samples t-tests and Wilcoxon Signed-Rank Test for Paired Samples between each 2 groups and surprisingly the results are as follows: T-test: Log G1-Log G2: Insignificant Log G1-Log g3: Significant Log G2-Log G3: Significant. Exactly the same results were achieved with Wilcoxon Signed-Rank Tests for Paired Samples. How can I explain this?
2- I tested the raw data (no Log). As a reminder, the means for Groups (G1-3) are 33.01, 31.20, and 22.93 and the variances are 129.10, 116.56, and 75.12 respectively. We have outliers (according to QQ plot) in G2 and G3 and negative normality results in G3. Since the assumptions for ANOVA are not met, I ran non parametric Kruskal-Wallis follows by Schaich-Hamerle Test and Contrast KW tests. Also I ran Welch’s Test and the result of all of the tests and post hoc test were insignificant. I even ran the resampling ANOVA test on 3 groups (bin size=0.5, Bins 0-6,Bootstrap, Iterations=10000) and p value>0.05, meaning that the samples are equal in their means. But out of curiosity, again I ran two paired samples t-tests and Wilcoxon Signed-Rank Test for Paired Samples between each 2 groups and while I expected not significance, the exact same results as was achieved with the Log values was attained: T-test and Wilcoxon signed-rank tests results : G1-G2: Insignificant G1-G3: Significant G2-G3: Significant.
Can you help me why the global tests (ANOVA and Kruskal-Wallis) contradict the post hoc paired tests and which one I should trust and report as my result? Is it possible to get significant results from parametric and non-parametric paired tests while the omnibus ANOVA was not significant? Am I dealing with false positive paired results?
P.S: Prior running the paired tests, the F tests were insignificant (P>0.05) for all the pairs and on both raw and Log10 data.
THANK YOU SO VERY MUCH IN ADVANCE,