paired t-test, re: sample size calculation

Question

We are performing a clinical intervention, that we believe will significantly improve some parameters in patients. We expect that the intervention will be successful 85% of the time (it will either be successful or not). We will be measuring the same parameters before and 3 months after the intervention. After that, we are planning to use paired t-test to determine whether the intervention results in a significant increase in these same parameters after the procedure compared to pre-procedure. We expect that if the intervention is successful (we know that the intervention is successful 85% of the time from preliminary studies), the parameters will improve to a certain degree, and we based our sample calculation on that effect estimate.

Specifically, I use pre-treatment (5000) and post-treatment (6500) means, pre and post group standard deviations (3000 each), correlation 0.5, alpha 0.05, power 0.95, one-sided (we expect an increase as indicated by prelim data. If there is a decrease, this well-known procedure would not be performed, thus one-sided). These numbers give me n=45 in STATA. After assuming a drop-out rate of 20%, I get n=54

However, I am not entirely sure how to incorporate the 85% procedural success rate into the sample size calculation. We know that the procedure will fail in approximately 15% of the patients, and they will not benefit from the procedure. In fact, there is a small chance (1-5%) that the parameters will get worse. The sample size calculation that I perform in STATA obviously does not factor in the procedural success rate.

I thought about multiplying the sample size by 100/85 to make up for the 85% procedural success. That is n=54 will be n=64.

Could you please tell me if this is acceptable? Is there a better way to do this?

Edit: After seeing the first reply, I felt I need to clarify: Our outcome variable is continuous. In sample size calculation, we are estimating that following the intervention, patients will be more physically active (measured by daily steps), and will have 1500 steps/day more (from say 5000 steps/day to 6500/day steps). However, with this estimated effect size, STATA output that I get (n=54), estimates that all participants on average will have to show this average effect size. On the other hand, if the procedure fails, daily steps will not improve. That is to say, if we enroll 54 patients, 54 x (0.85)= 45 patients will have successful intervention. If the 45 patients show +1500/day improvement, we will fall short of power. Therefore, I feel like an adjustment is needed in the sample size, for which I multiplied 54 with 100/85 (85% procedural success). I was meaning to ask if this adjustment (n x 100/85) would be an appropriate way to fix this issue.

Answer 1

I will see if I understand you correctly first, and explain my suggested approaches. Hereafter I will comment on your mentioned method.

Part 1 You are performing a clinical intervention on all included subjects.

As I understand, you research question is; is there an improvement from baseline to 3 month after treatment?

There are two ways to consider this.

(1) Your outcome measurement is continuous (each individual have a change from baseline to 3 month, that takes all (or most) values in an interval) (2) Your outcome is binary (each individual will end with a statement, improvement or no improvement)

In the first case, for each individual you will measure $x_0$ (baseline measurement) and $x_3$ (measurement after 3 months). For each individual you can then calculate the changes in the effect $d=x_3-x_0$. To calculate the sample size using paired t-test, you need to have a prior guess of the mean of $d$ and the standard deviation of $d$. Thus you need the mean difference rather than the means before and after. Of cause you also need to decide a power and a significands level together with a minimal relevant effect. The minimal relevant effect should be based on clinical considerations about how much change you want to be able to detect. In this case the success-rate is irrelevant, since you only consider the mean improvement.

The second approach you are interested in determining the proportion of improvement with a specific certainty.
In this case you have an expected improvement of 85%, and you need to give a margin of error, which is how precise you want to be able to estimate the proportion of improvement.

To the calculations above there is of cause some underlying assumptions that you should be aware of. However the internet should contain sufficient information, and including this would start me moving off topic.

Part two I will now comment on what you write, since I don't completely understand it.

In you text above you write

Specifically, I use pre-treatment (5000) and post-treatment (6500) means, pre and post group standard deviations (3000 each), correlation 0.5, alpha 0.05, power 0.95, one-sided (we expect an increase as indicated by prelim data. If there is a decrease, this well-known procedure would not be performed, thus one-sided). These numbers give me n=45 in STATA. After assuming a drop-out rate of 20%, I get n=54

First, you mention a mean before and after treatment. That is fine, but then you are not looking at paired t-test, but rather a t-test for comparing two populations, which is i think not want you are interested in.

Second, the correlation is a different measure, why are you suddenly including this?

You mention a STATA command, and it is possible that this command use some work-around, but since you did not mention it it was no possible for me to check what it actually does.

I hope this may help you.