6
$\begingroup$

I am interested in calculating the power for the beta effect size, for example

Call:
lm(formula = log1p(y) ~ x)

Residuals:
    Min      1Q  Median      3Q     Max 
-0.5684 -0.1881 -0.0413  0.1494  1.2312 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.59725    0.02460  24.279   <2e-16 ***
x           -0.06087    0.05514  -1.104     0.27    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.2551 on 1667 degrees of freedom
Multiple R-squared:  0.0007306, Adjusted R-squared:  0.0001312 
F-statistic: 1.219 on 1 and 1667 DF,  p-value: 0.2697

So, the beta (effect size) for x here is -0.06087. Can I calculate the power of detecting current effect size with two-tailed significant level of 0.05 in the following equation

Z(power)=abs(beta)/se-1.96=0.06087/0.05514-1.96=-0.85
Power=pnorm(Z(power))=0.196

So can I say with current sample size, I have 20% power to detect the significant associations between x and y with current effect size.

$\endgroup$
  • $\begingroup$ Maybe this is obvious for everyone else, but: power (in this context) refers to binary hypothesis tests (p values), not to the precision of the estimation of the coefficient/effect size. $\endgroup$ – jona Jun 22 '14 at 13:00
2
+25
$\begingroup$

The answer is "no" but not for the reason in vafisher's answer.

The correct formula for the power of a two-sided hypothesis test for a single regression coefficient is $$\begin{align} \mathrm{power}=&\operatorname{Pr}\left(t_{\mathrm{df}} \le -\frac{D}{\operatorname{se}\left[D\right]} - {t}_{\mathrm{df},\frac{\alpha}{2}} \right)+\\&\operatorname{Pr}\left(t_{\mathrm{df}} > -\frac{D}{\operatorname{se}\left[D\right]} + {t}_{\mathrm{df},\frac{\alpha}{2}} \right) \end{align}$$

where $D$ is the effect size, in this case $D=\hat{\beta}-\beta_0=\hat{\beta}$. This is given in the appendix of Dupont and Plummer (1998) (pdf), equation (A1) on p. 599, and in these notes.

The "Z" version of this test is just the normal approximation to the t-test. Note that the t-squared test (i.e. the $\chi^2$ test) is equivalent to the nested F test of regression models that differ by a single coefficient.

$\endgroup$
-1
$\begingroup$

Short answer, no. The t-statistic you're looking at is the square root of the F discussed in this post: What is the power of the regression F test? for the removal of a single independent variable.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.