Recently, I was tasked with a sample size calculation for a study in which the outcome is to be modeled using a Poisson regression (i.e. a generalized linear model). For quick and simple calculations of this nature, I often use PASS, a statistical software package dedicated to power/sample size calculations. However, I noticed that in the "Poisson Regression" menu for PASS, one of the options is specifying the distribution of the PREDICTOR variable (X). I was rather confused by this, since regression models don't make assumptions about the distribution of the covariates, so I checked the online documentation (here).

On page 870-2 of that document, they give a formula for sample size calculations. On that page, they state, "The variance [of the regression parameter estimate $\hat{\beta}$] for the non-null case depends on the underlying distribution of $X$."

I always learned rather unequivocally during my education that regression models do NOT make ANY assumptions about the distribution of the covariates themselves, and that even assumptions about the distribution of regression parameters are purely inferential (i.e. the parameter estimates themselves will be unbiased regardless of the underlying distribution of the covariate, assuming a properly specified model, but assumptions about the distribution of that parameter are useful for building confidence intervals, etc.). And it doesn't seem straightforward to me to draw a direct relationship between the distribution of a covariate and the variance of a corresponding parameter estimate, since the latter should be driven by the functional relationship between the covariate and the outcome.

The PASS documentation quoted above cites a 1991 Biometrika paper called "Sample Size for Poisson Regression", which I investigated in an effort to better understand what is going on here. That paper is available online here (I don't believe there is a paywall, but I'm on an institutional network at the moment so I may be wrong on that). As with the PASS documentation, this paper talks extensively about the maximum likelihood estimates for the regression parameters as functions of the distribution of the covariates.

At the bottom of page 1 of this paper, they write the likelihood function of the Poisson model (using their notation) as:

$L(\beta_0,\beta) = \prod_{i=1}^{N}f_X(x_i)f_T(t_i)\lambda_i^{y_i}exp(-\lambda_i)/y_i!$



This is, of course, not the usual likelihood function for a Poisson model, which would typically would not include $f_X(x)$ or $f_T(t)$ (where $f_X$ is the distribution of the covariates, $X$, and $f_T$ is the distribution of the exposure times (i.e. the 'offset' term in a Poisson model).

Granted, the paper notes that they are treating $X$ and $T$ as random variables, but I am struggling to understand why they are doing this, since it is so radically different from the traditional approach to estimating regression parameters using maximum likelihood.

This paper further cites a 1989 JASA paper that is making similar calculations for the case of logistic regression, again with the variance of parameters as a function of the distribution of the covariates themselves (and with a similar expression for the likelihood including some $f_X(x)$. Now THIS paper (available here) also includes a table (Table 1, on page 28), which seems to parameterize the distribution of X in terms of the regression parameters!

I am having a very difficult time understanding this. I have always learned that regression models do not make assumptions about the distribution of the covariates, or even of the regression parameters, yet it seems that these methods for sample size calculations under logistic regression and Poisson regression (both of these papers I linked to are fairly well cited) are explicitly making such assumptions.

Can anybody shed any light on this subject?

1) Do we ever need to make assumptions about the distribution of covariates in a regression model? Am I simply incorrect in believing we never need to make these assumptions, especially if we are assuming we have a correctly specified regression model?

2) If we DON'T need to make these assumptions, then what is the utility of doing so for the purposes of these sample size calculations? I will note further that using the formulas in these papers produces some RADICALLY different sample size estimates for the same effect sizes, based only on changing the assumed distribution of the covariates. I don't see how estimates using this method can be valid if we answer "NO" to question 1.

  • $\begingroup$ Continuing to research this question, it seems that the issue might have something to do with the fact that in the case of a non-linear link function, the covariance of the regression parameters does not have a closed form. I will try to find some good reference son this subject in the GLM literature to see if it sheds any light, but the properties of the link function may be the key reason it is necessary in the case of a Poisson/binomial regression but not, say, a traditional linear regression. $\endgroup$ – Ryan Simmons May 15 '17 at 22:12
  • $\begingroup$ I think you are heading down the wrong road with the issue of a link function. Simply log transforming an outcome in a linear regression is an instance of a GLM with gaussian variance and a log link. GLMs also specify a mean-variance relationship: which partially explains why the how the actual predicted values of the outcome affect the power above-and-beyond the "effect size" (as coined by Cohen). Poisson regression, for instance, uses a log link as well, but also a Poisson variance so that $Var(Y) = E(Y)$. $\endgroup$ – AdamO May 17 '17 at 20:11

You do not need to make assumptions about the distribution of the predictors $X$ in order to estimate the regression model or to make inference, but the variance of the obtained estimators will depend on aspects of the distribution of $X$, in particular, its spread as for example measured by its variance. This should be quite clear in the case of simple linear regression:

two linear regression models with different spread of x's

This shows a situation with simple linear regression, the same model ($y=1+x+\text{error}$), but in the left panel observation planned with a well-spread-out $x$, in the right panel the same number of observations planned, but with a more concentrated distribution of the $x$'s. It is intuitively clear that the design of the left panel will give better information for estimation of the model, and the algebra of simple linear regression confirms that.

In the following we will show this for simple linear regression, but the argument and conclusion is equally valid for multiple linear regression. The simple linear regression model is $$ y_i = \alpha + \beta x_i + \epsilon_i $$ and the usual assumptions of homoskedasticity and independent error terms. Then the estimator $\hat{\beta}$ of the slope can be written $$ \hat{\beta} = r_{xy} \frac{s_y}{s_x} $$ with the usual definitions. Its variance can be written $$ \DeclareMathOperator{\V}{\mathbb{V}} \V \hat{\beta} = \frac{\frac1{n-2}\sum \hat{\epsilon}_i^2}{\sum (x_i-\bar{x})^2} $$ and it is obvious from that formula that increasing the variance of the $x$' s will decrease the variance of $\hat{\beta}$. There is some subtle nuances here, in the usual theory for (simple) linear regression, the $x$'s are simply known constants, they are not random variables. Therefor the name design matrix, the values of $x$ are not observed values of a random variable, they are designed, that is, chosen, by the statistician. That reflect the origin of this terminology in design of experiments. In the majority of applications, probably, this is not the case, we do observe $x$ as values of some random variable. But still, inference is done conditional on the observed values. So, in this sense, since $x$ is not modeled as a random variable, it does not have a distribution! So, in that sense, inference cannot depend on something that does not exist (in the model). But, as the variance formula above shows, the variance of estimator $\hat{\beta}$ depends on the empirical variance of $x$.

So, if you are able to influence data collection, it is better to plan for well spread out values of $x$.

All of the above (except the concrete formulas) will be valid for Poisson regression. I will not repeat the arguments in that case.

  • $\begingroup$ Great answer! I definitely understand the logic, here. The only reason I am hesitant (for now) to accept it is the gnawing question of why we don't typically make these explicit assumptions about the distribution of X when doing power/sample size calculations for linear regression, when the papers I cited seem to make it necessary for the case of Poisson regression or logistic regression. Is there some specific difference between linear regression and these GLMs that make them more "sensitive" to such assumptions (e.g. due to non-linear link functions)? $\endgroup$ – Ryan Simmons May 16 '17 at 13:28
  • $\begingroup$ Or is this simply a case of inconsistent application, and that the typical methods of power/sample size calculations for linear regressions just gloss over the issue entirely? $\endgroup$ – Ryan Simmons May 16 '17 at 13:29
  • $\begingroup$ The point is that power calculation for linear regression can can be done explicitely (via non-central t or F distributions), so there is no need to specify a distribution for $X$, though, you will need the empirical varianc(es). If you do it via simulation, which can be practical, you will need to specify distributions for $X$, in some way. For Poisson regression (and other nonnormal glm's) there are no exact, explicit formula, so simulation proposes itself. You could find approximate formulas, though. Non-linearity of the link function doesn't play any role in this, i think. $\endgroup$ – kjetil b halvorsen May 16 '17 at 13:48
  • $\begingroup$ I will try to augment the answer when time ... $\endgroup$ – kjetil b halvorsen May 16 '17 at 13:49

You do not need to make assumptions about the distribution of covariates in order perform regression, but you do need to make assumptions about the distribution of covariates in order to simulate results of regression modeling. The latter is the point of performing a power analysis: to make a minimum number of reasonable assumptions to illustrate the feasibility of an analysis. It is more apparent when you calculate power using simulation, which is effectively the same thing as using MCMC integration to calculate quantiles of non-central distributions according to critical values for null hypothesis significance testing.

In general, the greater the variability of the covariates, the more power you have. That is why a stratified sample can improve power. A random sample of 100 people 65 and over will be result in very precise estimate of trends in people close to age 65 because of the distribution of age. Older ages will be undersampled giving these people highly influential values, possibly skewing analyses. On the other hand, If you sample 10 people in each 5 year age segment from 65 to 110, the estimates are equally precise in each age stratum, the distribution of age is considerably more variable, and the overall analysis is more efficient.

A power analysis should reflect the advantage of design strategies even if the same type of model is ultimately applied. Even in descriptive studies, design can significantly reduce margin-of-error. The most famous example of Gallup poll blowing national estimates out of the water with a mere 3,000 carefully picked people remains the most pertinent example. Read more here.

  • $\begingroup$ While there is some validity in the statement that you need to make assumptions about the distribution of covariates for simulations, I don't think that this leads to the general conclusion you need to make such assumptions for power analyses. If this were true, then you would need to a priori state the distribution of the covariates for ALL power analyses, which is something you don't need to do in any software package I'm aware of for linear regression. While I admit that appealing to the capabilities of software packages is not a statistically rigorous exercise, it is still meaningful. $\endgroup$ – Ryan Simmons May 15 '17 at 21:54
  • $\begingroup$ For example, quickly looking through the documentation on power analyses for multiple linear regression, it doesn't look like you need to make such assumptions in R, SAS, STATA, or PASS in order to make sample size/power calculations. Unless there is some additional property of traditional linear regression that makes it more robust to the distribution of covariates than, say, a Poisson regression, I don't think this answer holds. $\endgroup$ – Ryan Simmons May 15 '17 at 21:57
  • $\begingroup$ Further, it seems to me the rest of your post is off-topic, as it has more to do with selection/sampling bias than parametric assumptions about covariate distributions. $\endgroup$ – Ryan Simmons May 15 '17 at 22:04
  • $\begingroup$ If the software you have looked at for power analysis in li.ear regression do not ask for distribution of $x$ directly, then it must ask for such information indirectly, in some form. Otherwise itcannot calculate power at all $\endgroup$ – kjetil b halvorsen May 16 '17 at 5:04
  • 1
    $\begingroup$ @kjetilbhalvorsen consider inference about $\hat{\beta}$ using the Wald statistic: $W = \hat{\beta} / se(\hat{\beta})$ with a single covariate $X$. What is the expression for $se(\hat{\beta})$? $se(\hat{\beta}) = \sum_{i=1}^{n} r_i^2 / X^TX$. Increasing the variance of $X$ by design decreases $se(\hat{\beta})$ inverse proportionally. I would say the software is limited if it does not allow you to input $X$ of varying designs and you should use analytic power calculation or simulation. $\endgroup$ – AdamO May 16 '17 at 14:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.