2
$\begingroup$

When one applies interim sample size reestimation based on nuisance parameter estimates (observe fraction of initial sample size, estimate nuisance parameter(s) based on the obtained data, reestimate sample size based on interim estimates of nuisance parameter(s), and observe remainder of patients (if necessary), do final analysis based on all the data), is this a case of double dipping/p-hacking? I think not, because you not test the hypothesis at interim, you only estimate the relevant parameters. Is this correct?

$\endgroup$

1 Answer 1

3
$\begingroup$

My read is that this wouldn't be "p-hacking" per se because you're not adjusting your analysis plan based on the hypothesis test. However, I would buy that this might inflate Type 1 error (the issue with p-hacking), so it might be useful to treat this as an instance of sequential analysis and correct for the potential inflation: http://daniellakens.blogspot.com/2014/06/data-peeking-without-p-hacking.html

$\endgroup$
2
  • $\begingroup$ Isn't sequential analysis more for those designs where you actually use a stopping rule based on the treatment effect? I only look at the nuisance parameters and so there is not really a stopping rule, the sample size can increase or decrease after the interim 'look'. But just because I do not change my analysis plan based on the hypothesis test but based on the estimates of the nuisance parameters makes it not-p-hacking? $\endgroup$
    – gir emee
    May 24 at 14:52
  • 2
    $\begingroup$ That sure sounds like a stopping rule to me! $\endgroup$
    – whuber
    May 24 at 16:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.