0
$\begingroup$

In many research studies, researchers use measurement tools which produce scores that are all in integers. When such tools are applied to 2 groups of subjects, researchers use independent-samples t-test as a formal statistical test between the two groups.

(Note: these integer scores are not Likert scale data, RATHER these integers usually result from a test with $n$ items each dichotomously scored (i.e., ${0, 1}$), each subject's score is SUM of these $n$ items, which is necessarily an integer).

Question

Given the equality of population variances assumption for the t-test, how can I simulate the above t-test scenario while data are in the integer form?

What I have tried

I could think of this scenario as drawing from 2 binomial distributions with one binomial for "group 1" set to have a tiny bit higher mean compared to "group 2". But the problem with this set-up is that, the equality of population variances is violated. That is, increasing "group 1's" mean by just a bit increasing its probability of success will necessarily decrease "group 1's" variance compared to the variance of "group 2" and this violates the equality of variances assumption.

Here is my R code:

n1 = 30; n2 = 30 ; p1 = .52 ; p2 = .5 ; max.score = 15

y = as.vector(unlist(mapply(FUN = rbinom, n = c(n1, n2), size = c(max.score, max.score), prob = c(p1, p2))))

  groups = factor( rep(1:2, times = c(n1, n2)) )

cbind(m1 = mean(y[groups == 1]), m2 = mean(y[groups == 2]), sd1 = sd(y[groups == 1]), sd2 = sd(y[groups == 2]) )

(t.value = unname(t.test(y ~ groups, var.equal = TRUE)[[1]]) )
$\endgroup$
  • $\begingroup$ 1. "researchers use independent-samples t-test" ... well some researchers, sure, but many don't. 2. I don't think a binomial is a suitable model for the kinds of data you're talking about (presumably Likert scales ... sums of individual Likert items). Binomial will tend to have too little variation (and not enough flexibility of shape) compared to real data $\endgroup$ – Glen_b Jun 14 '17 at 5:36
  • $\begingroup$ @Glen_b, could you please let me know what would be suitable distribution in your view? So, this is not Likert scales data, these are SUM of ${0, 1}$ (i.e., dichotomousely scored) items from a test with $n$ items, so each subject's score is SUM of these ${0, 1}$ items which forms necessarily an integer. That's why I thought of a binomial distribution? $\endgroup$ – rnorouzian Jun 14 '17 at 6:02
  • $\begingroup$ Oh, okay. My apologies; yes, a binomial may be a reasonable first approximation if there's not a lot of variation in the probability of a 1 across items within a single total. The general case is Poisson-binomial, which in different situations may have one of several reasonable approximations. Another possible approach for modeling it is via is a quasi-binomial (though this is only suitable for fitting a model, not simulating one) $\endgroup$ – Glen_b Jun 14 '17 at 6:15
  • $\begingroup$ @Glen_b, I'm applied researcher, but obviously very interested in learning. Would you mind showing one of the approaches you're suggesting as a response? I deeply appreciate it. $\endgroup$ – rnorouzian Jun 14 '17 at 6:22
  • $\begingroup$ A simulation of a Poisson binomial is just like simulating a binomial but you let the p vary across trials (components); I can't really tell you what sets of $p$'s you should use. So for example for one observation you might have p=rep(c(.20,.24,.36,.55,.78),3);sum(rbinom(15,1,p)). The effort comes into figuring out how a whole vector of $p$'s will vary in your simulation. You might of course consider some continuous mixture distribution, such as a beta-binomial $\endgroup$ – Glen_b Jun 14 '17 at 6:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.