3
$\begingroup$

I have a data set with two groups. I have multiple trials with different Ns in each trial, and different numbers of trials for each group. In each trial, for each group, I am counting the number of successes.

How can I estimate the proportion of successes for each group under a binomial distribution (i.e. success or failure), with confidence intervals? How can I test whether the proportion of successes is the same between the two groups?

The standard Z-test for proportions is only described for one trial per group.

This question wasn't sufficient: Determining statistical significance of difference between two binomial distributions

Nor was this: Test if two binomial distributions are statistically different from each other

Improve this question
$\endgroup$
8
  • $\begingroup$ The N samples within a single trial are not perfectly independent, but it is fairly reasonable to assume independence. $\endgroup$
    – Dylan Richard Muir
    Jul 3, 2015 at 9:22
  • $\begingroup$ Do you have the outcomes for each individual in each trial (in each group)? $\endgroup$
    – Daniel
    Jul 3, 2015 at 9:29
  • $\begingroup$ @Daniel: Sorry if I misunderstand your question. For each of the N individual samples in each trial (in each group), I know the outcome -- either success or failure. $\endgroup$
    – Dylan Richard Muir
    Jul 3, 2015 at 9:33
  • $\begingroup$ But do you know which individual is responsible for each sample in each trial? e.g. if group A has 3 individuals (a, b, c), and the first trial for group A has 5 samples, do you have (1, 0, 1, 1, 0), or ({1, b}, {0, a}, {1, c}, {1, a}, {0, b})? $\endgroup$
    – Daniel
    Jul 3, 2015 at 10:57
  • $\begingroup$ Individuals are only sampled once, and are not repeated across groups or across trials. Individual ID can therefore be neglected: group A trial 1 is (1, 0, 1, 1, 0). $\endgroup$
    – Dylan Richard Muir
    Jul 3, 2015 at 14:24

1 Answer 1

Reset to default
2
+50
$\begingroup$

If I'm understanding your question right (and a couple simple assumptions are met), the second link you have gives you what you're looking for. So a simple probability fact is that if X and Y are independent, X~Bin(n,p) and Y~Bin(m,p) then X+Y~Bin(m+n,p). That means that if the trials in each of your groups are independent then

$\sum_{i=1}^{|G_{1}|}X_{i}$ ~Bin($\sum_{i=1}^{|G_{1}|}N_{i}^{1}$, p) and $\sum_{i=1}^{|G_{2}|}Y_{i}$ ~ Bin($\sum_{i=1}^{|G_{2}|}N_{i}^{2}$,p)

where the G's are the groups, the N_i are the n for each binomial trial (superscript tells the group) and |G| is the size of G. From there, you're back at the problem of estimating proportions and confidence intervals in binomial families. The big things to make sure of are that the trials within groups are independent and that you have no reason to believe the probabilities of success are different between trials within groups.

Improve this answer
$\endgroup$
4
  • $\begingroup$ And what if the probability of success differs between trials within groups? For example, if there are differences in how difficult it is to detect "success" between trials? $\endgroup$
    – Dylan Richard Muir
    Jul 13, 2015 at 9:40
  • $\begingroup$ Using the same kind of logic as above (binomials as sums of bernoulli's) you can treat your sums as poisson binomial's. They're not particularly nice but at least you have a distribution for them. en.wikipedia.org/wiki/Poisson_binomial_distribution $\endgroup$
    – Nick Thieme
    Jul 13, 2015 at 14:12
  • $\begingroup$ Are there any tests that would operate on P.B. distributed sets of values? Or would one have to roll-your-own monte carlo test? $\endgroup$
    – Dylan Richard Muir
    Jul 13, 2015 at 15:01
  • 1
    $\begingroup$ Here's a link to a better explanation than I could ever give:stats.stackexchange.com/questions/93645/… $\endgroup$
    – Nick Thieme
    Jul 13, 2015 at 15:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.