7
$\begingroup$

When teaching GLMs, it's commonly taught that the linear and log link are collapsible but that the logit is not. The log link is collapsible because the curves are proportional and the means are modeled with ratios. The linear link is collapsible because the trend-lines are parallel and the means are modeled with differences. For a general link function $g$, is there a way of expressing two quantities (means) $X$ and $Y$ to know whether $g$ is collapsible or not?

$\endgroup$
  • 2
    $\begingroup$ -> could you provide the definition of "collapsible"? $\endgroup$ – Lucas Roberts Jan 22 '18 at 21:39
  • 2
    $\begingroup$ @LucasRoberts At a 10,000 ft view, collapsibility means that adjusting for other covariates which are uncorrelated with the main covariate will not change its estimated value (although it may change the SE). $\endgroup$ – AdamO Jan 24 '18 at 16:14
9
$\begingroup$

Yes, there is, for instance, you can find one in Pearl, definition 6.5.1. Consider any functional $g[P(x, y)]$ of the joint distribution of $X$ and $Y$. We say $g$ is collapsible on $Z$ if:

$$E_z\left[g[P(x, y|z)]\right] = g[P(x, y)]$$

We can see that collapsibility must make reference to not only $P(Y,X)$ but at least two more things: (i) a measure of association of $X$ and $Y$; and, (ii) a third variable $Z$. Note that if $g[P(x, y|z)]$ is constant across Z, the definition reduces to checking $g[P(x, y|z)] = g[P(x, y)]$, Greenland and Pearl call this simple collapsibility (see reference in the end). We will use this case below, since it reflects the case of the coefficient changes you refer to.

With this in mind, we see that we need to assess when certain measures are collapsible with respect to certain variables --- and while the collapsibility of some measures might vary depending on the specific parameterization of the joint distribution, we can still say something about the collapsibility of some measures even without specifying $P(Y, X)$.

For instance, let's take the logit link. The logit coefficients aim to estimate odds ratios, so to better understand the collapsibility of the logit coefficients we can investigate the collapsibility of the odds ratio.

Let's write the Z specific odds ratio as:

$$ \text{OR}(yx|z) = \frac{P(y_1|x_1, z)p(y_0|x_0, z)}{p(y_1|x_0, z)p(y_0|x_1, z)} $$

Suppose that it is constant across $Z$. Then we would say the z specific odds ratio is (simply) collapsible over $Z$ if $\text{OR}(yx|z) = \text{OR}(yx)$. Note that if $Y \perp Z|X$ then:

$$ \text{OR}(yx|z) = \frac{P(y_1|x_1, z)p(y_0|x_0, z)}{p(y_1|x_0, z)p(y_0|x_1, z)} =\frac{P(y_1|x_1)p(y_0|x_0)}{p(y_1|x_0)p(y_0|x_1)} = \text{OR}(yx) $$

Thus the odds ratio is collapsible when $Z$ is independent of $Y$ given $X$. Therefore, we should expect the coefficient of $X$ in a logistic regression not to change when adding $Z$ in this case.

Since the odds ratio is symmetric, if $X \perp Z|Y$ we also have collapsibility of the odds ratio, that is:

$$ \begin{align} \text{OR}(yx|z) &= \frac{P(y_1|x_1, z)p(y_0|x_0, z)}{p(y_1|x_0, z)p(y_0|x_1, z)} =\frac{P(x_1|y_1, z)p(x_0|y_0, z)}{p(x_1|y_0, z)p(x_0|y_1, z)} \\ &=\frac{P(x_1|y_1)p(x_0|y_0)}{p(x_1|y_0)p(x_0|y_1)} = \text{OR}(yx) \end{align} $$

We can also see that we should not expect it to be collapsible over $Z$ with the condition $X \perp Z$ only. Thus, if $Z$ is independent of $X$ given $Y$ we should expect the coefficient of $X$ in a logistic regression to be unchanged by the inclusion of $Z$, but if we can only guarantee $Z \perp X$ we should not expect that.

Note we are talking about when the associational measure Z specific odds ratio of $X$ and $Y$ is collapsible over $Z$. You could estimate $P(y|x,z)$ with other (non) parametric methods other than a GLM with logit link. If you then compute the Z specific odds ratio, what we said above about when we should expect it to be collapsible would still be valid, regardless of how $P(y|x,z)$ was obtained. It is true, however, that we are providing sufficient conditions here, and specific parameterizations of the joint distribution might have cases where we should not expect a measure to be collapsible but in that case it is.

As to the risk ratio, assume it is constant across $Z$ and write the Z-specific RR as:

$$ \text{RR}(yx|z) = \frac{P(y_1|x_1, z)}{P(y_1|x_0, z)} = k $$

Thus $P(y_1|x_1, z) = kP(y_1|x_0, z)$. If $X \perp Z$:

$$ \begin{align} \text{RR}(yx) &= \frac{P(y_1|x_1)}{P(y_1|x_0)} = \frac{\sum_{z}P(y_1|x_1, z)p(z|x_1)}{\sum_{z}P(y_1|x_0,z)P(z|x_0)}\\ &= \frac{\sum_{z}P(y_1|x_1, z)p(z)}{\sum_{z}P(y_1|x_0,z)P(z)} = \frac{\sum_{z}kP(y_1|x_0, z)p(z)}{\sum_{z}P(y_1|x_0,z)P(z)}\\ &= k\frac{\sum_{z}P(y_1|x_0, z)p(z)}{\sum_{z}P(y_1|x_0,z)P(z)} = k = \text{RR}(yx|z) \end{align} $$

Thus the constant Z-specific RR would be collapsible with $X\perp Z$ and, for example, we would not expect the log-link coefficient of $X$ to change by adding $Z$ in this case.

As a final remark, when $X \perp Z$, even though the Z specific odds ratio is not collapsible, the odds ratio using standardized probabilities would still be collapsible, since

$$ \sum_{z}P(y|x, z)P(z) = \sum_{z}\frac{P(y,x, z)}{P(x|z)} = \sum_{z}\frac{P(y,x, z)}{P(x)} =\frac{P(y,x)}{P(x)} = P(y|x) $$

In fact, $Y \perp Z |X$ or $X\perp Z$ are sufficient conditions for the collapsibility of standardized measures. You might find this paper by Greenland and Pearl useful.

$\endgroup$
  • 1
    $\begingroup$ I have seen this before, but what puzzles me is that I do not know how to use this definition to prove that the log link is collapsible but that the logit is not. (the latter is proved easily with a counter example, but a general proof is the same problem). $\endgroup$ – AdamO Jan 24 '18 at 16:17
  • $\begingroup$ Hmm.. in your definition you call the functional $g$ collapsible. The odds ratio is not a functional, it is the exponentiated coefficient for a linear model using the logit link functional of the proportion/risk. Is my understanding right? What is collapsible? I have heard the link function is what we refer to as collapsible (or not) based on changes in the association measure (OR, or RR) $\endgroup$ – AdamO Jan 29 '18 at 15:35
  • 1
    $\begingroup$ @AdamO just complemented the answer, see what you think. $\endgroup$ – Carlos Cinelli Feb 1 '18 at 3:09
  • $\begingroup$ I have been mulling over your excellent answer for a while now. You say $Y \perp Z|X$ or $X \perp Z$ are sufficient conditions for collapsibility, but this does not hold with logistic regression. Do you rather mean the probability measures assigned to those values by the regression model as you clarified earlier? And if so, how do we quantify $Z \perp X$? $\endgroup$ – AdamO Feb 23 '18 at 23:16
  • $\begingroup$ @AdamO I wouldn't say it's the probability measure assigned by the fitted model because $X \perp Z$ is true or false regardless of the model you pick. But if I understood correctly what you asked, I think you're kind of right, in the sense that independence is a strong condition, some models will require less than that. For instance, zero (partial) correlation is sufficient for collapsibility in linear regression. So X and Z may be dependent, yet the linear regression still collapsible. Does that make sense? $\endgroup$ – Carlos Cinelli Feb 24 '18 at 2:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.