I have metabolomics data, I only have 5 paired samples. the data does not follow a normal distribution so instead of doing a t test I was thinking of doing Mann Whitney Wilcoxon (MWW) test but I have read that for the MWW test the data has to be independent and this is not the case. Which test can I use?
Thank you
There are three main issues (independence, normality, and sample size), which I would like to address separately--to the extent possible.
Independence. If you are doing a paired test, you are essentially doing a one-sample test on the difference in values for each pair. Both the one-sample t test and one-sample Wilcoxon (signed-rank) test require that the differences are randomly sampled from a population, and hence that differences are independent of one another. (Of course, the two observations that yielded the differences for each pair are not independent, but that is an issue.)
Normality. The t test requires normal data in order to give accurate results. The Wilcoxon test does not require normal data. However, for straightforward interpretation of results, it is best that the differences be symmetrically distributed. (There are several measures of 'location', mostly all giving the same result for symmetrical data.) From small samples alone, it can be difficult to know whether data are normal.
Sometimes prior knowledge about the kind of data at hand can be an important clue. In particular, standard tests of normality have poor power detecting non-normality in small samples. Looking at boxplots and normal probability plots can give useful, but subjective, impressions.
The difference between two exponential distributions with equal rate has a Laplace distribution, which is symmetrical, but far from normal because of fat tails--both left and right. The following simulation in R shows that, among Laplace samples of size $n=10,$ the Shapiro-Wilk test detects non-normality with probability only about $0.15.$
set.seed(818)
pv = replicate(10^5, shapiro.test(rexp(10)-rexp(10))$p.val)
mean(pv <= 0.05)
[1] 0.15156
Sample size. You say you have only $n=5$ pairs. With a sample of size five, a two-sided Wilcoxon test can never give a P-value below $1/16 = 0.0625 > 5\%.$ A one-sided Wilcoxon test gives P-value $1/32 = 0.03125 < 5\%,$ only if all five paired differences have the same sign (as appropriate to the direction of the alternative).
wilcox.test(1:5, alt="greater")
Wilcoxon signed rank test
data: 1:5
V = 15, p-value = 0.03125
alternative hypothesis: true location is greater than 0
Suppose that data are five differences from a Laplace distribution with center at $\mu=\eta=3$ and variance 2. Notice that the effect is relatively large.
Below are results from a one-sided t test (inappropriate because data are not normal). The power against this large effect is about $96\%.$ For a Wilcoxon signed-rank test, power is about $0.88.$
set.seed(2020)
pv = replicate(10^5, t.test(rexp(5)-rexp(5)+3, alt="greater")$p.val)
mean(pv <= 0.05)
[1] 0.96265
pv = replicate(10^5, wilcox.test(rexp(5)-rexp(5)+3, alt="greater")$p.val)
mean(pv <= 0.05)
[1] 0.88063
The actual significance level of the one-sided test (when $\mu=\eta=0),$ at a nominal $5\%$ level, is about $3\%$ (Laplace data are far from normal, so the t statistic doesn't have a Student t distribution). For Wilcoxon, the actual significance level is about $4\%$ (the test statistic for $n=6$ is highly discrete). [R code is the same, except omit the +3s.]
Tentative conclusion. Of course, results for your (presumably non-Laplace) data may be quite different. However, if you have a large effect and if it is appropriate to do one-sided tests, you might get useful results using a Wilcoxon signed-rank test on paired differences.
If you were inclined to do a ttest but the assumptions of normality aren't met (but the data points are independent and exchangeable) you can still do a permutation test. Its the non-parametric analog to the ttest that doesn't care about the distribution of the data. Here is a shell of the code to do that:
rm(list=ls(all=T))
set.seed(1)
#import the data
setwd('')## set your working directory here
dat=read.csv('',as.is=T)## call in your data in csv format here
#calculate the mean
tmp=aggregate(variable~sample,data=dat,mean)
#calculate the difference in means
obs.stat=tmp$variable[1]-tmp$variable[2] #this is our observed statistic
n=nrow(dat) #total number of observations
nsim=1000 #total number of simulations (i.e., randomizations)
sim.stat=rep(NA,nsim) #empty vector to store the simulated statistic from our randomizations
for (i in 1:nsim){ #do this shuffling "nsim" times
print(i)
prod=sample(dat$sample) # mix
#create a new dataset
dat.new=data.frame(variable=dat$variable,sample=prod)
#calculate mean based on this new dataset
dat.new1=aggregate(variable~sample,data=dat.new,mean)
#calculate the difference in means based on this new dataset
sim.stat[i]=dat.new1$variable[1]-dat.new1$variable[2] #store the simulated statistic
}
#create a histogram of the simulated statistics.
#This is the distribution of our test statistic under the null hypothesis (i.e., null distribution)
hist(sim.stat,main='Simulated difference in means',xlab='')
#is the observed test statistic likely to have come from this null distribution?
abline(v=obs.stat,col='red')
#what proportion of simulated statistics was smaller than the observed statistic?
vec=sim.stat < obs.stat #query: is sim.stat smaller than obs.stat?
mean(vec) #proportion of simulations that satisfy the above query. This is the p-value of our randomization test.
#Notice that numbers close to 0 or close to 1 indicate extreme outcomes.
#For instance, a p-value of 0.999 indicates that almost all of the simulated statistics were
#smaller than the observed statistic. As a result, this suggests that we can reject the
#null hypothesis even though our p-value was greater than the threshold of 0.05.
If something like this is what you are after, I am happy to clarify anything if you get stuck.
Good luck, E123
