(Non-parametric) test for similarity between three or more vectors of observations?

Question

I have three vectors each of six observations. I imagine other situations will arrive in which I might have more than three vectors with other lengths. I want a statistical test to compare them.

>x
          a    b    c       d    e    f 
          0    4    27470   9    20   1 

>y
          a    b    c       d    e    f
          0    20   27823   6    17   0 

>z
          a    b    c       d    e    f
          1    43   6653    666  762  0 

To me, it seems pretty obvious that the last vector, z, is different from the two other ones because the dand ecomponents are much larger both numerically and percentage-wise.

I have no idea which distributions the observations might follow (they're counts of how many customers belong to one of six possible groups), so I guess the test should be non-parametric.

I have tried using a Kruskal-Wallis test, but it returns a very un-significant p-value:

kruskal.test(list(x,y,z))

        Kruskal-Wallis rank sum test

data:  list(x, y, z)
Kruskal-Wallis chi-squared = 0.71944, df = 2, p-value = 0.6979

I am not sure if it is my intuition that fails or if the test isn't suitable for this type of comparison.

Are there other types of tests that I can try? I know the practice of trying different tests until one returns the desired output is questionable, but I'm pretty sure in this case the null should be rejected.

Can anybody suggest other tests that are suitable for this situation?

Thanks!

Answer 1

You should make use of all information you have.

1) You have counts, i.e., your data can probably be expected to be Poisson distributed (under/overdispersion might be an issue).

2) You measured different groups and these groups are the same in all samples.

I would use a generalized linear model here:

DF <- data.frame(count = c(0,4,27470,9,20,1,0,20,27823,6,17,0,1,43,6653,666,762,0),
                 group = rep(letters[1:6], 3),
                 treat = rep(c("x", "y", "z"), each = 6))

fit1 <- glm(count ~ group + treat, data = DF, family = poisson)
anova(fit1, test = "Chisq")

#Analysis of Deviance Table
#
#Model: poisson, link: log
#
#Response: count
#
#Terms added sequentially (first to last)
#
#
#      Df Deviance Resid. Df Resid. Dev  Pr(>Chi)    
#NULL                     17     230330              
#group  5   210344        12      19986 < 2.2e-16 ***
#treat  2    14183        10       5803 < 2.2e-16 ***
#---
#Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

#test for over/underdispersion
library(AER)
dispersiontest(fit1, trafo = 1)
#Overdispersion test
#
#data:  fit1
#z = 1.7742, p-value = 0.03801
#alternative hypothesis: true alpha is greater than 0
#sample estimates:
#  alpha 
#536.8357 

fit2 <- glm(count ~ group + treat, data = DF, family = quasipoisson)
anova(fit2, test = "F")

#Analysis of Deviance Table
#
#Model: quasipoisson, link: log
#
#Response: count
#
#Terms added sequentially (first to last)
#
#
#      Df Deviance Resid. Df Resid. Dev       F    Pr(>F)    
#NULL                     17     230330                      
#group  5   210344        12      19986 43.3825 1.844e-06 ***
#treat  2    14183        10       5803  7.3129   0.01104 *  
#---
#Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

As you see, your "vectors" can clearly be considered different, i.e., you have a significant treatment effect.

One might also be interested in interactions between treatments and groups, but you lack the replicates needed to investigate that.