1
$\begingroup$

I have a situation where I have two groups, a Treated population and a Control group. We are looking to find a statistically significance difference in the occurrence of some event X between the two groups. In other words, does X happen more or less often for the Treated population over time than the Control population.

My problem is that the Treated and Control group memberships evolve over time. It's not possible for a Treated member to switch to Control (or vice versa), but the study involved new members joining Treated or Control over time. How can I test for the statistical significance of X happening over time if there are new members participants the groups during the study?

Do you have to throw out everyone except those that were initially in the two groups, or is there a type of statistical significance test that accounts for adding new participants over time?

Thanks!

Example data

TREATED

  • N (initial) = 150,000
  • N (after month 1) = 160,000
  • N (after month 2) = 165,000
  • N (after month 3) = 180,000
  • N (final) = 185,000

CONTROL

  • N (initial) = 50,000
  • N (after month 1) = 55,000
  • N (after month 2) = 60,000
  • N (after month 3) = 62,000
  • N (final) = 77,000
$\endgroup$
  • $\begingroup$ Are the participants who started initially different from those who started in a later month? If so, on what rationale do you include the participants who joined after the initial group? If not, why not just wait until you have the final N to do your test? (For example, are you planning to make sequential decisions on whether to continue or stop the study?) $\endgroup$ – Joel W. Oct 31 '15 at 23:48
  • $\begingroup$ Yes, the participants who join in later months are different than the initial participants, although the initial participants continue to be a part of the study. This is a marketing problem where the participants are customers who meet certain criteria. The marketing is an ongoing campaign where the customers are sent flyers, emails, etc. All of the participants are those that qualify for this effort, and we hold back a random percentage of customers who qualify for the campaign to use as a control group. We want to know if the marketing is having an impact to decide if we should continue it. $\endgroup$ – Anon Nov 2 '15 at 14:53
0
$\begingroup$

Look into work on "stopping rules". A stopping rule is "A procedure that allows interim analyses in clinical trials at predefined times, whilst preserving the Type I error at some pre-specified level" per http://community.cochrane.org/glossary/5#letters. Some history of this field is provided here: http://onlinelibrary.wiley.com/doi/10.1002/sim.6472/full.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.