A priori publication of statistical analysis plan (SAP) for clinical trials

Question

This is somewhat of a regulatory question, rather than statistical question. But I assume that the readership of this community will be the most knowledgeable on this matter.

Question:

Is there any regulatory agency that actually documents submission of a statistical analysis plan (SAP) for clinical trials prior to the final clinical database lock and treatment unblinding?

What surprised me a bit is that there appears to be no obligation to actually submit a statistical analysis plan a priori. Could someone please tell me that I am incorrect about this and that a priori SAPs are always available to the public?

Here are some examples of rather vague suggestions:

ICH E9 Statistical Principles for Clinical Trials:

"The plan should be reviewed and possibly updated as a result of the blind review of the data and should be finalised before breaking the blind."

U.S. FDA, Guidance for Industry Developing Medical Imaging Drug and Biological Products, June 2004:

"We recommend that sponsors [...] submitted to the protocol before images have been collected."

Specific Background:

Recently, a "positive" phase III trial result has been published for a disease in my area of expertise. However, the are several "red flags" in my opinion:

• Actual results are not reported
• Only mixed model for repeated measures (MMRM) estimates were reported, but it is rather unclear what co-variates were used for "adjustment" in the model [on clinicaltrials.gov, there is no mentioning of "adjustment" for co-variates]
• The outcome variable was transformed [on clinicaltrials.gov, there is mentioning of transformation of the outcome variable]

Having worked on natural history data for this disease before, it is obvious that one can easily find "significant" between-group differences by iterating across combinations of outcome transformations (no transformation, sqrt, log10) and co-variates combinations (age, gender, baseline lesion size). For obvious reasons, I suspected that a statistical analysis plan must be deposited somewhere a priori. However, the specific study did not publish the statistical analysis plan on clinicaltrials.gov.

Answer 1

Focusing on Phase 3 studies only:

1. SAPs are submitted to regulatory authorities along with the the protocol, informed consent documents, letters of intent, etc. etc. etc. for approval prior to the first patient initiated. The regulatory authorities may reject the submission if the SAP is not clearly aligned with the objectives.

2. SAPs may be made publicly available after the database lock or publication of top-line results. There are many examples on clinicaltrials.gov.

3. When submitting the CSR for regulatory approval by the respective agencies in accord with the ICH E9 guidance, the SAP is a controlled document meaning the cumulative revision history and the timeline are kept, and should match with the protocol. You cannot modify the SAP after database lock (OK you can but it's 99.9% likely to result in a rejection). If any aspect of the CSR is not in alignment with the SAP it is a major finding.

4. It makes sense that the number of examples are relatively few. The authorities are hesitant to mandate that SAPs (or protocols even) are made publicly available. This is because the authorities are perceived as an immense hurdle in the drug approval process - a necessary one at that. I believe the authorities feel that if "successful" (meaning led to approval) study examples were too widely circulated, others would so closely copy the template they would fail to ever present or challenge new ideas, even when necessary. Indeed, you often hear that the FDA "won't approve a drug without a blinded controlled study" or "hates R and loves SAS" or "doesn't approve Bayesian trial designs but only frequentist". The opposite is true. But these new ideas need to be defensible to a number of highly experienced, cross-functional, and educated agents.