Completely confused about alpha and p values, help please? I'm trying to work thru the following question. I'm pretty sure A doesn't apply, but having trouble with the concept in general.
A study team compared study drug x with placebo for maintenance therapy of a certain type of cancer. The primary outcome was progression-free survival (PFS). The a priori $\alpha$ was set at 0.025. The authors reported a p value of 0.02 for the primary outcome. Which is the best interpretation of this p value? 
A. There is a 2% probability that chance alone is responsible for prolonged PFS with drug x. 
B. This p value is statistically significant and assumes the difference in PFS between the two groups would be as large as or larger than observed no more than 2.5% of the time if only chance were creating the discrepancy. 
C. The authors should reject the test hypothesis because this p value is less than α. 
D. The null hypothesis has a 2% probability of being true because the p value is less than α. 
 A: Let's break the statements down after going through what alpha level and p-value mean. Alpha level is a threshold that someone wants a p-value to "pass" in order to reject the null-hypothesis (relates to C). The p-value means: Given the data observed, the model, and assuming the null-hypothesis is true, it is the probability of getting a test-statistic (t or F usually) larger than the 1 found if you were to sample again an infinite amount of times. However, in frequentism, we aren't allowed to state specific probabilities that a hypothesis is true or false (relates to D). Reread my definition of p-value, NOTHING is stated about chance or randomness (relates to A & B), as nothing is caused by chance or randomness. Chance and randomness are our states of knowledge, we may not know what caused something to take the number it did or perform the action under certain conditions, but "chance" or "randomness" can't be a cause.
Given all of this, I would go with C. However, it's not the answer that's important, but the concepts, definitions, and reasoning.
