2
$\begingroup$

I am trying to calculate the minimum detectable effect size (MDE) of an explanatory variable in a multiple linear regression.

The regression looks like the following: $$ y_i = \beta_0 + \beta_1X_1 + \beta_2*X_2 + \beta_3*X_3 + \beta_4*X_4 + \epsilon_i $$ whereby I am only interested in the MDE of $X_1$ in this multiple linear regression.

I have found code for linear regression in R: pwr.f2.test().

However, I am not sure if this can eliminate the minimum detectable effect size for one variable. Does anyone know how to get the minimum detectable effect size of one explanatory variable in a multiple linear regression? How can I implement this in R?

$\endgroup$
3
$\begingroup$

Here is a useful formula to remember:

$$\pm \beta_{j}^{a}=\frac{\left(z_{1-\alpha / 2}+z_{\gamma}\right) \sigma_{y \mid \mathbf{x}}}{\sigma_{x_{j}} \sqrt{n\left(1-\rho_{j}^{2}\right)}}$$

Here, $\beta_j^\alpha$ is the minimal detectable effect for a covariate assuming:

  • Our FPR is $\alpha$ and our desired power is $\gamma$. If you use the popular $\alpha=0.05$ and $\gamma=0.8$ then $z_{a-\alpha/2} \approx 1.96$ and $z_\gamma \approx 0.84$.

  • The residual variation is $\sigma_{y\vert x}$. This is akin to the standard deviation of the residuals assuming the linear model is the true model.

  • The standard deviation of the covariate is $\sigma_{x_j}$. If the covariate is continuous, this can always be rescaled to 1 through standardization. If the covariate is binary then you can use $\sigma_{x_j} = \sqrt{f(1-f)}$ where $f$ is the fraction of your sample with the covariate set to 1.

  • $n$ is the sample size

  • $1/(1-\rho_j^2)$ is the variance inflation factor for the covariate (which can be hard to estimate a priori).

$\endgroup$
4
  • $\begingroup$ Can you maybe cite a paper or explain where this formula comes from? $\endgroup$
    – Laura
    Jun 30 at 14:41
  • $\begingroup$ @Laura Sure. Formulas come from the end of chapter 4 of this book. Citations and explanations are included therein. $\endgroup$ Jun 30 at 14:44
  • $\begingroup$ Thank you! Do you by any chance also know if there is a way to implement it in R (despite calculating the MDE by hand)? $\endgroup$
    – Laura
    Jun 30 at 15:09
  • $\begingroup$ @Laura it should be very easy to program yourself, I don't know of any functions or libraries. $\endgroup$ Jun 30 at 15:32
1
$\begingroup$

Along the lines of Demetri's answer, the approach greatly simplifies finding MDE in a multivariate model - a point of confusion for many early analysts. The $\sigma_{y|x}$ that is expressed is conditional on all the other x. In other words, there is practically no difference between finding power in a multivariate model and in a bivariate model: inference for the model coefficient boils down to knowing the conditional variance of the Y, and subtracting off all the degrees of freedom for adjusting the other $X$; it is just a t-test.

So proceed by considering MDE for a t-test, supply the conditional variance of $Y$ adjusting all the other covariates which is required as a free-parameter in the power/sample size calculations, then subtract 1 from the overall $n$ for each adjusted variable.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.