When one applies interim sample size reestimation based on nuisance parameter estimates (observe fraction of initial sample size, estimate nuisance parameter(s) based on the obtained data, reestimate sample size based on interim estimates of nuisance parameter(s), and observe remainder of patients (if necessary), do final analysis based on all the data), is this a case of double dipping/p-hacking? I think not, because you not test the hypothesis at interim, you only estimate the relevant parameters. Is this correct?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
My read is that this wouldn't be "p-hacking" per se because you're not adjusting your analysis plan based on the hypothesis test. However, I would buy that this might inflate Type 1 error (the issue with p-hacking), so it might be useful to treat this as an instance of sequential analysis and correct for the potential inflation: http://daniellakens.blogspot.com/2014/06/data-peeking-without-p-hacking.html
-
$\begingroup$ Isn't sequential analysis more for those designs where you actually use a stopping rule based on the treatment effect? I only look at the nuisance parameters and so there is not really a stopping rule, the sample size can increase or decrease after the interim 'look'. But just because I do not change my analysis plan based on the hypothesis test but based on the estimates of the nuisance parameters makes it not-p-hacking? $\endgroup$– gir emeeCommented May 24, 2022 at 14:52
-
2$\begingroup$ That sure sounds like a stopping rule to me! $\endgroup$– whuber ♦Commented May 24, 2022 at 16:51