Mann Whitney Wilcoxon Heterocedasticity

Question

I am currently working with some data with 11 cases; with a quantitative variable and a factor (treatment). My data in R are as follows:

Q <- c(0.52257929, 0.59409435, 0.59990197, 0.43122963, 0.54950869, 0.53396515, 0.59540344, 0.07396450, 0.55989774, 0.04496018, 0.35368660)
Treatment <- c("C", "A", "C", "A", "C", "A", "C", "A", "A", "A", "A" )
df <- data.frame(Q, Treatment)

I am trying to apply the Mann-Whitney -Wilcoxon test (based on ranks) to test whether the data from both treatments belong to the same population, given that the sample size is small. One of the assumptions of this test is homoscedasticity; so I have applied the Fligner test, which I understand is more robust to non-normal data. In these data I find heteroscedasticity, and after trying the log and square root transformations, I have found that the difference in variance remains. I don't know what the next step is. I know I can also use permutation tests but the heterocedasticity assumption stays the same. Any advice is welcome. Thanks in advance!

Answer 1

Your main problem is that you have only $n_1 + n_2 = 7 + 4 = 11$ observations altogether. That's near the lower limit of sample sizes that can give signif. results for a Wilcoxon test. If you go ahead with the Wilcoxon test, in spite of obvious difficulties, the P-value is 0.055.

A one-sided Welch t test, which does not require equal variances, is (barely) significant with P-value 0.045, but it is fair to ask whether data are normal.

A one-sided permutation test using Welch t as metric is (barely) significant with P-value 0.048. However, one could question whether samples of such different variability are exchangeable. I don't show code and results for fear someone might take it as serious evidence of a difference.

You might do further P-hacking to get a smaller P-value, but the bottom line is that you don't have enough data--or enough separation between samples--to make a convincing case that C's tend to exceed A's.