【About this thread】:

This thread is divided from the following threads:
What is the post-hoc power in my experiment? How to calculate this?
I cut out for when Welch's correction is not required from the thread mentioned above.

【What I do not want】:(Please read carefully):

  • Criticism of the "post-hoc power" is no thank you !!
    This is not a place to criticize the"" post-hoc power." (This is also not a place to defend the "post-hoc power"".)

  • Definitions that do not use mathematical expressions are not welcome.
    Many editorials have only verbal explanations. There are many editorials without formulas. I'm sick of these.

Whenever a topic about post-hoc power comes out, there will be many people criticizing it. But they don't seem to like using mathematical formulas. Here is where we discuss statistics. Therefore, there should be no discussion without writing the definition in mathematical formulas.

I'm sick of mass-produced editorials without formulas. Do not repeat the same thing. It is written in mass-produced editorials written in words instead of mathematical formulas.

【Summary of my questions】

Please explain how to calculate the post-hoc power in the t-test without Welch correction using mathematical formulas. The following paper describes this. It would be helpful if you supplemented the part that I didn't understand.


【Prepare for My questions】
Although many many editorials criticizing post-hoc power is mass-produced, very few of then have descriptions based ion mathematical formula. Followings are briefs of above-mentioned paper.

enter image description here (Equation 01)

Here, the α is given in advance and, here
enter image description here (Equation 02)
enter image description here (Equation 03)

And, may be use the following d for δ,
enter image description here (Equation 04)

【My questions】

  • (My question1): What is the distribution followed by T '? Here, T' is the test static of Equation 1 and defined by the following (Equation 05).  ( Referring to the ordinary definition of power, T 'or T'-δ probably follows a non-central t distribution.But, in this paper, the word "non-central t distribution" never appears.)
    enter image description here (Equation 05)

  • (My question2): What is this ${Z}_{\alpha /2}$? Zα is the upper α point of which distribution? As an explanation of another formula in this paper," ${Z}_{\alpha /2}$is the upper α / 2 quantile of the standard normal distribution", why is the normal distribution suddenly appearing? Is the upper α/2 point t-distribution?

  • (My question3):  The power calculated by Equation 1 is written as Pow. At this time, is " β: = 1-Pow" "the probability that the second type of error has occurred in this experiment"?

  • (My question4): In the following 【My Experiment】, does 【My Code】 calculate the post-hoc power of Equation 1?

【My Experiment】

We randomly divide 20 animals into two groups, Group A and Group B. After that, for Group A, Foods A are fed, and for Group B, Foods B are fed. After a certain period, bodyweight was measured, and the data were as follows.

Group_A :40.2, 40.4, 40.6, 40.8, 41.0, 41.2, 41.4, 41.6, 41.8
Group_B :30.1, 30.3, 30.5, 30.7, 30.9, 31.1, 31.3, 31.5, 31.7, 31.9, 32.1

I would like to conduct a two-sided t-test without using Welch's correction with a significance level of 0.05 to see if there is a significant difference between the two groups.

【My Code】

#Load data
Group_A = c(30.2, 30.4, 30.6, 30.8, 31.0, 31.2, 31.4, 31.6, 31.8)
Group_B = c(30.1, 30.3, 30.5, 30.7, 30.9, 31.1, 31.3, 31.5, 31.7, 31.9, 32.1)

# Welch Two Sample t-test
t.test(Group_A,Group_B, var.equal=T )


cd = cohen.d(Group_A, Group_B)

pwr.t2n.test(n1 = 9, n2= 11, d = cd$estimate, sig.level = 0.05, power = NULL,
         alternative = c("two.sided"))

【P.S.】 I'm not very good at English, so I'm I'm sorry if I have some impolite or unclear expressions. I welcome any corrections and English review. (You can edit my question and description to improve them)


2 Answers 2

  1. I don't see a T.
  2. I guess they're using Z instead of t.
  3. The definition of power is $ 1 = \beta $. Equation 1 is for the power of a test given an effect of a certain magnitude in the population. I don't think you can talk about the probability of a type II error in your experiment, if you don't specify an effect size to detect.
  4. Yes.

I would not describe those equations as the formulas for power. Power is calculated using non-central distributions, e.g. https://en.wikipedia.org/wiki/Noncentral_t-distribution But I don't think you need to understand the calculation of a non-central distribution to understand power (or post hoc power).


For the formulae, your best bet is to get a copy the Cohen book referenced in the documentation for the pwr.t2n.test function: Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale,NJ: Lawrence Erlbaum.

Your code looks okay, but you may need to check the documentation for cohen.d to make sure it's doing what you want and to see if there are options that may make a difference in the result. That is, there are different ways to calculate Cohen's d.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.