# How to choose a Power and Alpha combination for a fixed sample size?

Let's say I have a sample size established with alpha=0.05 and power=0.8 (based on time constraint for study).

The same sample size can be achieved with any of:

alpha = 0.001 and power = 0.31
alpha = 0.01 and power = 0.58
alpha = 0.25 and power = 0.95
alpha = 0.999 and power = 0.998


I want to know is what levels of risk to expect from the study of this size: what alpha I can aim for and what power I can hope to achieve.

Which combination of alpha and power do I adopt? Why?

One way to think about it is that if you plot power versus alpha, you obtain what is in essence an ROC curve (receiver operating characteristic). Typically, ROC curves are used to evaluate the capability of a diagnostic procedure such as cancer screenings. They weigh the probability of a true positive (i.e., power) against the probability of a false positive (I.e., alpha). An overall measure of the capability of the diagnostic test is the area under the ROC curve.

As for what alpha and power to choose, that depends on the relative consequences of attaining a false positive versus a false negative. The greater the relative cost of a false positive, the smaller the alpha you should choose.

• But I don't understand the consequences. My problem is it seems the same sample has different alpha/power profiles SIMULTANEOUSLY. How is a study for which I arbitrarily chose alpha 0.05 any different from the hypothetical exact same study with same sample but alpha 0.01, since I effectively just converted that alpha to 0.05 by increasing power. Or to put another way, how can two researchers run a shared study with the same hypothesis and get one dataset, but one says "significant" while the other says "not even close", just based on their own arbitrary choice of an alpha/power combination.