# Is a p-value correction necessary for assessing pairwise location shifts on centered data?

I am currently assessing whether or not a location-shift can be assumed in non-parametric comparisons to be able to formulate the rejection of the null hypothesis in other terms than the probabilistic index. To do so, I center my data and execute a pairwise two-sample Kolmogorov-Smirnov test on each pair. The data is in some cases unbalanced and not-normally distributed.

Currently I am using this function (in R):

pairwise.ks.test<-function (x, g, p.adjust.method = p.adjust.methods, alternative="two.sided",centered=T,...)
{
DNAME <- paste(deparse(substitute(x)), "and", deparse(substitute(g)))
g <- factor(g)
METHOD <- if (centered)
"Pairwise KS test on centered data"
else "Paiwise KS test "
compare.levels <- function(i, j) {
xi <- x[as.integer(g) == i]
xj <- x[as.integer(g) == j]
ks.test(xi, xj, alternative=alternative, ...)$p.value } compare.levels.centered<-function(i,j) { xi <- x[as.integer(g) == i]-mean(x[as.integer(g) == i],na.rm=T) xj <- x[as.integer(g) == j]-mean(x[as.integer(g) == j],na.rm=T) ks.test(xi, xj, alternative=alternative, ...)$p.value
}

if(centered)
PVAL <- pairwise.table(compare.levels.centered, levels(g), p.adjust.method)
else PVAL <- pairwise.table(compare.levels, levels(g), p.adjust.method)
ans <- list(method = METHOD, data.name = DNAME, p.value = PVAL,
class(ans) <- "pairwise.htest"
ans
}


And at the moment I apply it on my list of datasets without p-value correction:

lapply(datalist,function(x)pairwise.ks.test(x$value,x$trt,p.adjust.method="none",alternative="two.sided",centered=T,exact=F))


My set has ties and therefore an exact p-value cannot be calculated (hence exact=F). As I only want to assess a possible location shift for each pairwise comparison to be able to formulate rejection of $H_0$ in terms of medians or means in Holm-corrected pairwise Wilcoxon-rank sum tests, should I also apply a (Holm?) p-value correction to assess multiple location shifts?

• I'm not sure I understand - doesn't centering remove the very effect you're testing for? – Glen_b Dec 11 '13 at 0:24
• @Glen_b I center the data to see if there is a location shift: after centering I can use a two-sample KS test to see if data CDFs differ significantly. If they don't that means that there was a location shift (e.g by subtracting a location parameter the CDF's are equal). – FM Kerckhof Dec 11 '13 at 11:28
• "If they don't that means that there was a location shift" -- no, it doesn't necessarily mean that. Unless there's something I missed, your logic is faulty in several ways at once. – Glen_b Dec 11 '13 at 14:38
• I'm sorry, @Glen_b but I do not see my faulty logic here. $H_0$ is that there is a location shift. i.e. $H0$: $CDF_1$=$CDF_2$ - $\Delta$. If after substracting a location parameter (either median or mean, doesn't really matter under $H_0$) under $H_0$ the CDFs should be the same. Hence they will not be significantly different. Hence you cannot reject $H_0$ in favor of $H_1$. Hence we withold equality of centered CDFs. Hence the location shift assumption holds. Or am I missing something here? – FM Kerckhof Dec 12 '13 at 16:17
• Here's a few problems to get you started: (1) failure to reject doesn't mean $H_0$ (and the other assumptions) are true. That is, if you assume a location shift, subtract an estimate of one, and fail to reject a goodness of fit test, it doesn't lead to the conclusion that it was a location shift, it may be only that you lacked power to pick up any other difference. (2) The KS is based on a completely specified distribution. It doesn't have the desired properties when you do what you did. Your nominal p-values don't take account of what you did to the data; they're less likely to reject. – Glen_b Dec 13 '13 at 3:14

• Thanks, @Matt.Vn this was exactly my reasoning. I don't want to loose too much power whilst still being able to express rejection of the $H_0$ in terms of location parameters as the PI is a challenging thing for people to deal with. – FM Kerckhof Dec 11 '13 at 11:30