Power analysis for linear model test asks for an analytical expression of the power of a linear model. The answer claims that

Note that this depends on the correlation between $X$ and $T$, $r_{TX}$.

In practice, when running the power analysis of an experiment, prior to the experiment, you don't really know this (the treatment has not been assigned yet, you don't have data labelled on treatment and control). At most, you have historical data with the covariate X, the outcome Y and a way to randomize the treatment.

How do we calculate $r_{TX}$ in practice? Do we simulate a specific randomization and calculate power with that randomization? Do we simulate many and average the power? Is there a standard way of doing this?

Also, how do we estimate $\sigma$ and $\sigma_T$ in practice?

  • 1
    $\begingroup$ I don't think there is a standard way. I always recommend going with a "reasonable" simulation, based on knowledge about potential correlations. This has the advantage of allowing sensitivity analyses - if your power depends strongly on the specific value of this correlation, this is a valuable piece of information. $\endgroup$ Mar 27 at 9:42
  • 1
    $\begingroup$ How are you going to assign treatment to subjects? If it's at random, why would you expect the covariate X and the treatment assignment to be correlated? $\endgroup$
    – dipetkov
    Mar 27 at 10:01
  • $\begingroup$ The idea is to replicate the assignment that is going to be done in the experiment. I understand that in some cases it would be 0, but other cases (cross-over trials, clustered designs, stratified designs) this could be non-0. Is your suggestion to assume that the value of r_tx is 0? $\endgroup$ Mar 27 at 11:24
  • $\begingroup$ I think that, in general, assuming that it is 0 will give you over-confident results, as when you analyze the data of the experiment it may be close to 0, but non-0 $\endgroup$ Mar 27 at 11:25
  • 1
    $\begingroup$ I understood this question to be about a particular experiment. It seems rather it's more of a theoretical question since you'd like to consider so many different experimental designs (cross-over trials, clustered designs, stratified designs). Thank you for the clarification. $\endgroup$
    – dipetkov
    Mar 27 at 11:41

1 Answer 1


As Stephan Kolassa said in a comment, simulation is the best way to proceed. With modern computing technology it's reasonably straightforward to use your understanding of the subject matter to evaluate a wide range of scenarios.

Analytical formulas for power size date back to times when data crunching was done by hand on mechanical calculators. That was before digital computers were commercially available, long before electronic calculators like those produced by Wang or Hewlett-Packard became available to individuals in the 1960s, and even longer before personal computers extended major computing capacity to a wide audience. At that time those analytical power formulas were developed, that was about the best one could do.

Although analytical power formulas provide general guidance, the large level of uncertainty about the values of the parameters in the formula means that it's wise to evaluate a range of possibilities in study design. See this page, for example, about the difficulty simply in evaluating the value of a linear-regression error $\sigma^2$ from a limited data set. You want to be covered in case your initial estimate is incorrect.

Simulation allows you to handle more complicated situations like the crossover, stratified, and clustered designs that your mention in a comment. If your outcome isn't a simple continuous value as in a linear regression, say a binomial or survival outcome, simulation is even more likely to prevent you from being led astray.

The simulations provide information about the types of results that you might end up finding. You then have to apply judgment to choose the sample size based on that information.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.