(Non-parametric) test for similarity between three or more vectors of observations?

I have three vectors each of six observations. I imagine other situations will arrive in which I might have more than three vectors with other lengths. I want a statistical test to compare them.

>x
a    b    c       d    e    f
0    4    27470   9    20   1

>y
a    b    c       d    e    f
0    20   27823   6    17   0

>z
a    b    c       d    e    f
1    43   6653    666  762  0


To me, it seems pretty obvious that the last vector, z, is different from the two other ones because the dand ecomponents are much larger both numerically and percentage-wise.

I have no idea which distributions the observations might follow (they're counts of how many customers belong to one of six possible groups), so I guess the test should be non-parametric.

I have tried using a Kruskal-Wallis test, but it returns a very un-significant p-value:

kruskal.test(list(x,y,z))

Kruskal-Wallis rank sum test

data:  list(x, y, z)
Kruskal-Wallis chi-squared = 0.71944, df = 2, p-value = 0.6979


I am not sure if it is my intuition that fails or if the test isn't suitable for this type of comparison.

Are there other types of tests that I can try? I know the practice of trying different tests until one returns the desired output is questionable, but I'm pretty sure in this case the null should be rejected.

Can anybody suggest other tests that are suitable for this situation?

Thanks!

• For kruskal-wallis it is recommended that sample size for each group be at least 5 otherwise the pvalue may not be accurate. Here you have only 3 samples per group. Reference Sep 20, 2019 at 19:52

You should make use of all information you have.

1) You have counts, i.e., your data can probably be expected to be Poisson distributed (under/overdispersion might be an issue).

2) You measured different groups and these groups are the same in all samples.

I would use a generalized linear model here:

DF <- data.frame(count = c(0,4,27470,9,20,1,0,20,27823,6,17,0,1,43,6653,666,762,0),
group = rep(letters[1:6], 3),
treat = rep(c("x", "y", "z"), each = 6))

fit1 <- glm(count ~ group + treat, data = DF, family = poisson)
anova(fit1, test = "Chisq")

#Analysis of Deviance Table
#
#
#Response: count
#
#Terms added sequentially (first to last)
#
#
#      Df Deviance Resid. Df Resid. Dev  Pr(>Chi)
#NULL                     17     230330
#group  5   210344        12      19986 < 2.2e-16 ***
#treat  2    14183        10       5803 < 2.2e-16 ***
#---
#Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

#test for over/underdispersion
library(AER)
dispersiontest(fit1, trafo = 1)
#Overdispersion test
#
#data:  fit1
#z = 1.7742, p-value = 0.03801
#alternative hypothesis: true alpha is greater than 0
#sample estimates:
#  alpha
#536.8357

fit2 <- glm(count ~ group + treat, data = DF, family = quasipoisson)
anova(fit2, test = "F")

#Analysis of Deviance Table
#
#
#Response: count
#
#Terms added sequentially (first to last)
#
#
#      Df Deviance Resid. Df Resid. Dev       F    Pr(>F)
#NULL                     17     230330
#group  5   210344        12      19986 43.3825 1.844e-06 ***
#treat  2    14183        10       5803  7.3129   0.01104 *
#---
#Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


As you see, your "vectors" can clearly be considered different, i.e., you have a significant treatment effect.

One might also be interested in interactions between treatments and groups, but you lack the replicates needed to investigate that.

• Thanks for your very useful answer. The data comes from tabulating variables from a database. Which replicates would I need to investigate interactions between treatments and groups? Jun 12, 2015 at 12:46
• It's difficult to say. I'm more familiar with ecological research, where is is obvious how you would obtain replicates. Maybe you could repeat the tabulation over different time periods (e.g., week 1, week 2, ...), show that there is no time trend and uses the time periods as replicates? Even better would be independent replicate databases, but that's probably not feasible. Jun 12, 2015 at 12:57