3
$\begingroup$

I'm analysing a published logistic regression with 11 predictors, $\textrm{logit}(Y) = \sum \beta_i X_i$. The study unfortunately only reports which of the $X_i$ are significant predictors, and does not give any estimates of $\beta_i$.

Only $X_1$ is of interest to me. I need to try to show that the study is underpowered to detect effects from this predictor. I am comfortable running simulations to determine the power of the regression to detect different levels of $\beta_1$.

My problem is that the simulation would seem to require me to guess effect sizes for the other 10 predictors, $\beta_2$, ..., $\beta_{11}$. Most of these have not been studied in the prior literature, so I have no idea of what effect sizes to put in.

I feel like I must be missing some simple concept here, as power analysis for multiple logistic regression does seem to be a thing. What does it mean to run such an analysis? Does one guess $\beta_2$, ..., $\beta_{n-1}$? Or search over a range of values for them? Or is there some reason why the power for $\beta_1$ is insensitive to the values of the other $\beta_i$? (I can see why that would happen for linear regression, but not logistic.)

$\endgroup$

1 Answer 1

4
$\begingroup$

I need to try to show that the study is underpowered to detect effects from this predictor.

You can't determine the power of a study post hoc, but you can determine the power for similar studies that might be run in the future.

Given a $\beta$ (which is not the beta provided by the study. This would be a post hoc power calculation, and there are multiple threads here demonstrating why this is a bad idea) and a sample size, the statistical power is

$$ \gamma=1-\Phi\left[z_{1-\alpha / 2}-\left|\beta_j^a\right| \sigma_{x_j} \sqrt{n p(1-p)\left(1-\rho_j^2\right)}\right] $$

Here

  • $z_{1-\alpha/2}$ is the $1-\alpha/2$ quantile of a standard normal distribution. When $\alpha=0.05$ then $z_{1-\alpha/2} \approx 1.96$
  • $\sigma_x$ is the standard deviation of the predictor of interest
  • $n$ is the sample size
  • $p$ is the marginal prevalence of the outcome, and
  • $\rho$ is the multiple correlation of the covariate of interest with the other covariates.

So in principle, you don't need to know the log odds ratios for the remaining predictors -- you only need to know the multiple correlation between the predictor of interest and the remainder.

That doesn't sound easy, and granted it isn't, but the takeaway is that big $\rho$ means small power. Were your covariate orthogonal to the remainder -- because it was randomly assigned, as an example -- then $\rho$ would be 0 and you would have optimal power.

So what do you do when you don't know $\rho$? You could assume it is zero. This gives you the best case scenario, and you should operate under the assumption that the power is lower than what you've calculated.

If you're trying to achieve a minimum statistical power, I might compute a range of power calculations under various assumptions of $\rho$, and maybe use previous data to get an estimate of the multiple correlation.

So you don't need to know the coefficients for the remaining variables, just how strongly correlated your covariate is with the remaining ones.

$\endgroup$
6
  • $\begingroup$ Thank you! Could I have a source for the formula? $\endgroup$
    – Mohan
    Commented May 24 at 7:48
  • 1
    $\begingroup$ @Mohan Vittinghoff, Eric, et al. "Regression methods in biostatistics: linear, logistic, survival, and repeated measures models." (2005). See the end of the logistic regression chapter $\endgroup$ Commented May 24 at 13:36
  • $\begingroup$ Something I find surprising is that the power formula does not use the intercept from the GLM. I would’ve thought that with logistic regression, as the intercept became larger (more positive), the proportion of the whole sample that had a ‘YES’ dv would tend towards 100% percent, & consequently you would need a larger sample to compensate for that. $\endgroup$
    – Mohan
    Commented May 24 at 17:56
  • $\begingroup$ @Mohan the marginal probability plays that role $\endgroup$ Commented May 24 at 21:06
  • 1
    $\begingroup$ @Mohan it’s the probability of the outcome, ignoring all covariates $\endgroup$ Commented May 25 at 4:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.