Task:
I have e.g. 1 control group and 7 experimental groups which are treated differently. All groups have a dependent binomial variable: 1 = success, 0 = no success.
I want to conduct sequential tests, which version is "performing" better, between the control group and the experimental groups and stop the tests, when one of the experimental group is better that the control group (or it is proven, that the Null hypothesis is true).
Problem:
So in my opinion two problems arise: alpha I error inflation by sequential testing and alpha I error inflation by testing multiple hypothesis (per timepoint)
My research so far:
I found group sequential methods, Alpha-Spending approaches and SPRT to solve the sequential testing problem. And I read the sequential updating approach of Bayesian Methods in the book of Kruschke.
For the multiple hypothesis testing (per time), there are some techniques which control the overall alpha error rate (per time), like Bonferroni, Hochberg's step-up procedure and others.
But I haven't found how to combine the methods. Nevertheless, group sequential methods as well as Alpha-Spending approaches seem to be a bit inflexible. SPRT und Bayesian Updating seem to fit best for the need to have a small sample number, being able to test after every participant and not to have a predefined endpoint of the study.
Bayesian updating (with ROPE), as far a I know, does not have something like a predefinable alpha error rate (and no correction for multiple variables?). SPRT also does not support multiple variables. Please correct me if I am wrong. Do you have any suggestions where to dig deeper?
Best Regards Andreas