How to calculate probability of success based on prior p-value

Question

The FDA often requires a sponsor to conduct multiple clinical trials prior to approval. Given the following observations in a ph2 and ph3 trial, how would you go about predicting the probability of the next ph3 trial resulting in a statistically significant outcome on the primary endpoint.

Assume that aside from the number of patients that the ph2 and ph3 trials are the same.

In the ph2 trial of 90 patients on drug and 90 patients on PBO, the drug showed a benefit on the primary endpoint with a p-value of 0.017.

After the positive ph2 result, a ph3 trial was conducted. In the first ph3 trial 150 patients were on drug and 150 patients were on pbo. The ph3 trial the drug showed a benefit on the primary endpoint with a p-value of 0.025.

From this information how would you calculate the probability that a second larger ph3 trial with 300 patients on drug and 300 patients on pbo would result in a positive outcome on the primary endpoint?

Answer 1

The P-values can be interpreted as probabilities, but those probabilities relate to the observed data and the statistical model. They do not tell you the probability that the tested drug is effective and nor do they tell you the probability of a successful trial in the future.

To put a probability on the success of a future trial you would need a prior probability distribution on the effect size and a power function for the future trial design. In practice you might do without the prior and settle for an effect size point-wise interpretation of the power function in the manner of "if the true effect size is $x$ then we would have a $y$ percent chance of obtaining a significant result at the alpha level of $z$ if we have sample size $n$".

The power function (power as a function of effect size) is determined by the variance of the measured effect, the nature of the test, the acceptable alpha level, and the sample size. Notice that the P-values are not used. (Of course, in the planning of real clinical trials the statistical model might use advanced features like mixed effects and an adaptive sampling plan.)

Answer 2

I think your best plan is to use a meta-analysis program to combine the results from both trials and come up with an overall effect size (with confidence interval) and an overall P value.

This approach does not predict the next experiment, but it combines the current results which is probably what you really want to do.

I'd focus more on the effect (how much difference did the drug make and how certain are we about it) than the P value.