We ran a test marketing campaign at my company to see if our new customer segmentation has an impact on the response rate to our offer.

We have created two groups to run this test marketing campaign:

  • Group 1: 2,000 people randomly selected among existing clients.
  • Group 2 : 10,000 people that were selected following the new segmentation process.

For each of this group, we obtained the following number of people who responded to the campaign:

  • Group 1: 3 people out of 2,000
  • Group 2: 5 people out of 10,000

So we are dealing with a really rare case type of problem. My goal is to understand it there is a significant difference between the response rate of group 1 and group 2.


1) Can I run a t-test with such a rare case type of problem?

I suspect that this is not possible because the data is highly skewed (see this paper: http://advan.physiology.org/content/34/3/128)

2) Which test (possible to be ran with R) should I use to compare the mean of my two groups?

3) Is there a rule of thumb to determine if the t-test is not appropriate?

Any help appreciated!

  • 1
    $\begingroup$ You should not in general use t-tests on count data like this; there are tests suitable for comparing binomial proportions. $\endgroup$ – Glen_b Jan 27 '16 at 7:19

Fisher's exact test on this table:

                |   I  |  II  |
Response  | Yes |    3 |    5 |
          |  No | 1997 | 9995 |

The t-test will fail very far from 50-50. The chi-square test will fail with sparse counts. Fisher's will work (in R, this is fisher.test).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.