2
$\begingroup$

What would be the best data analysis to use for the following data? I was thinking of using the Wilcoxin ranked sum but there are so many ties. I have two independent groups and I am just looking to see if the counts differ between the treatment and control conditions.

    Count Condition
       2 Treatment
      36 Treatment
       1 Treatment
      26 Treatment
      11 Treatment
       0 Treatment
      69 Treatment
       5 Treatment
       0 Treatment
       4 Treatment
       1 Treatment
      19 Treatment
       4 Treatment
       0 Treatment
       1 Treatment
      69 Treatment
       2 Treatment
      11 Treatment
      58 Treatment
      12 Treatment
       0 Treatment
       0   Control
      10   Control
       0   Control
      42   Control
      13   Control
      14   Control
       0   Control
      52   Control
      26   Control

Thank you so much!

$\endgroup$
1
  • $\begingroup$ Can you say more about what these are counts of and explain the circumstances. There are a number of models specifically for counts that might be used but it's impossible to choose between them with no information. (However, you're very likely not going to identify a significant different between groups here unless there's a deal more information missing from the post that has a substantive effect) $\endgroup$
    – Glen_b
    Jun 11, 2019 at 4:18

2 Answers 2

2
$\begingroup$

Descriptive statistics and boxplots of your data do not look promising for finding a significant difference. Counts overlap considerably between Treatment and Control groups.

x1 = c(2,36,1,26,11,0,69,5,0,4,1,19,4,0,1,69,2,11,58,12,0)
x2 = c(0,10,0,42,13,14,0, 52,26)
sort(x1)
 [1]  0  0  0  0  1  1  1  2  2  4  4  5 11 11 12
[16] 19 26 36 58 69 69
sort(x2)
[1]  0  0  0 10 13 14 26 42 52

x = c(x1,x2);  g = c(rep(1,21), rep(2,9))
summary(x1);  summary(x2)
     Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
     0.00    1.00    4.00   15.76   19.00   69.00 
     Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
     0.00    0.00   13.00   17.44   26.00   52.00 

 boxplot(x~g, col="skyblue2", pch=20)

enter image description here

A Welch t test finds no significant difference. This test may not be valid because sample sizes are small and data (especially in the Treatment group) do not seem to be normal. Also, a two-sample Wilcoxon test shows no significant difference. This test returns warning messages about ties; nevertheless, its large P-value may be roughly correct.

t.test(x~g)$p.val 
[1] 0.8366992


wilcox.test(x~g)

      Wilcoxon rank sum test with continuity
      correction

data:  x by g
W = 88.5, p-value = 0.802
alternative hypothesis: true location shift is not equal to 0

Warning message:
In wilcox.test.default(x = c(2, 36, 1, 26, 11, 0, 69, 5, 0, 4, 1,  :
  cannot compute exact p-value with ties

A permutation test using the Welch t statistic as metric, has P-value about 0.84. The permutation test does not assume the data to be normal and is not ruined by the ties.

set.seed(610)
t.obs = t.test(x~g)$stat
t.prm = replicate(10^5, t.test(x~sample(g))$stat  )
mean(abs(t.prm) >= abs(t.obs))
[1] 0.83722

Here is a histogram of the simulated permutation distribution along with the observed value of the Welch statistic (vertical bar).

hist(t.prm, prob=T, br = 50, col="skyblue2")
abline(v=t.obs, col="red", lwd=2)

enter image description here

Note: (1) What to say in a formal report of analysis? Opinions differ, but for a mainly non-statistical audience I might show boxplots and report P-values of the Welch t and Wilcoxon rank sum test (0.84 and 0.80, respectively) and provide a note to mention briefly doubts about normality and ties. Say that a permutation test (with reference but no details), which is not subject to these doubts, gives P-value 0.84. So it seems unnecessary to look further for significance. (2) Search Internet for 'permutation test' for more information, perhaps including Eudey (2010).

$\endgroup$
3
  • 1
    $\begingroup$ Since these are counts, why not try a count-based model, like quasipoisson? $\endgroup$ Jun 10, 2019 at 18:17
  • $\begingroup$ @gung: Thanks much for finding typo. Fixed it and subsequent results. // Traditional analysis taking counts to be Poisson might be to use square roots of individual counts to 'stabilize' variance. But Welch t takes care of possibly different variances. Is there another Poisson-based method that should be provided as a link? $\endgroup$
    – BruceET
    Jun 10, 2019 at 22:32
  • $\begingroup$ That's true, @BruceET, these analyses may well be fine. My default on seeing counts is to use a count model, though. I might use a negative binomial w/o even thinking about anything else. A slightly simpler analysis for this thread might be a quasi-Poisson GLM. $\endgroup$ Jun 11, 2019 at 11:34
2
$\begingroup$

The counts look over dispersed so I'm going to jump right to a negative binomial model.

Call:
MASS::glm.nb(formula = count ~ group, data = d, init.theta = 0.4053622108, 
    link = log)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.7517  -1.1621  -0.4610   0.2324   1.2307  

Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)   2.8590     0.5296   5.398 6.72e-08 ***
grouptest    -0.1014     0.6332  -0.160    0.873    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Deviance goodness of fit test shows the model is a good fit. Confidence interval for test group is (-1.48 1.07). Not sure if that means anything to you (I would need to know what the data are measuring to preoperly interpret), but the CI seems relatively symmetric about 0, so for every argument that goes "The ci covers an important effect size of x" one could also argue "it also covers -x".

So yea, I don't think there is an effect here. The data seem really small here tho. Did you do a power analysis?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.