How to hand-calculate alpha error given power, effect and sample size in t-test in comparison of two independent group means for 2sided hypothesis?

Question

I am looking to understand alpha error calculation given power, sample size, and effect in a two-sided independent mean comparison. So the hypothesis is:

H0: mean1 = mean2 H1: mean1 != mean2

Assume we know the power, sample sizes, means and standard deviations for each group. From standard deviations, we can calculate a pooled standard deviation, and given means and pooled deviation, we can calculate effect size. Then, from there we can calculate non-centrality parameter.

I am good until this point. But from then on, for a two-tailed t-test, I am not sure how to go about reaching critical t value and find alpha.

I am looking into either an explanation or any type of resource that helps me calculate this by hand. I am aware that I can do this via G*Power:

1. Selecting t-test family
2. Means:Difference between two independent means (two groups)
3. Type of power analysis: Criterion - Compute required alpha-given power, effect size, and sample size.

However, there is no explanation step by step how this is calculated for a two-sided test and I am very confused. I would like to understand this better.

Any help is appreciated.

Answer 1

The alpha is not calculated. Instead it is a setting of the hypothesis testing procedure that is set in advance of the analysis. It is not data-dependent and is chosen by the analyst.

It sounds like you should be looking for background reading on the topics of statistical testing. Try these posts and papers:

• What is the difference between "testing of hypothesis" and "test of significance"?
• Interpretation of p-value in hypothesis testing
• Bad Statistical Practice in Pharmacology (and other basic biomedical disciplines): you probably don't know P
• A reckless Guide to P-values

Answer 2

Alpha can be solved by central and non central t distributions generated. Noncentral t distribution affects the beta (type II error) and central t distribution affects the alpha (type I error). Given sample sizes for control and treatment, as well as power and means, a greedy algorithm can solve for t critical, which can then be converted via central t distribution, into a alpha error probability value.