Estimating conditional effect of logistic regression

Question

I'd like to examine the conditional effect of a new variable, say $\mathbf{Z}$, on a logistic regression previously fit to my response vector $\mathbf{Y}$ using the predictor(s) $\mathbf{X}$.

Say that the logistic regression has already been fit to the predictors $\mathbf{X}$ and that I can't/don't want to change the coefficients associated with $\mathbf{X}$. (The dataset used to fit this logistic regression was "better" in some sense than the dataset that follows, but did not contain $\mathbf{Z}$.)

I'd now like to add $\mathbf{Z}$ to my set of predictors. (Although the dataset I'm using is "worse", I still feel it's good enough to estimate the effect of $\mathbf{Z}$.) But I only want the effect of $\mathbf{Z}$ on $\mathbf{Y}$ given by current predictions based on $\mathbf{X}$.

I know that, since these models are nested, I can use the Deviance to conduct a hypothesis test of whether or not to include $\mathbf{Z}$ in the new model (under the assumption that the coefficients on $\mathbf{X}$ could be changed), but what about the actual parameter estimates?

In the linear regression case, I could simply fit a regression using $\mathbf{Z}$ to the residuals of the model based on $\mathbf{X}$ but this is problematic in the logistic regression setting, since the residuals don't exist in their usual sense.

I have been performing an optimization by minimizing the log-loss with respect to $\mathbf{Z}$ while keeping the coefficients on $\mathbf{X}$ constant. This workaround doesn't give me the standard errors I'd like, although I can repeat this procedure on subsamples of the data to get an estimate of this (assuming I have enough data).

Is there an alternative (better) way to do this?

Answer 1

You can include terms with fixed coefficients using an offset. Technically, an offset is a predictor with coefficient fixed at 1, so you will first need to create a new variable that has the linear combination of the $X$'s with coefficients estimated from the first model.

Model for first data set: $$logit(P(Y=1)) = \beta_1X_1 + \dots + \beta_kX_k,$$ giving estimates $\hat\beta_i$.

Model for second data set: $$logit(P(Y=1)) = \gamma Z + 1\cdot(\hat\beta_1X_1 + \dots + \hat\beta_kX_k),$$ giving estimate of $\hat\gamma$ with standard errors and all the inference.

Most software for generalized linear models (such as logistic regression) has a way to include an offset. It is most commonly used for Poisson regression, but as you can see, it can be useful for other situations as well.

Here is how this would work in R:

set.seed(456724)
# first data set
dd1 <- data.frame(X1=rnorm(50), X2=rnorm(50), Z=rnorm(50))
dd1$Y <- with(dd1, rbinom(50, size=1, p=1/(1+exp(-2-X1+2*X2-Z))))

# first model fitted with only X1 and X2
mod1 <- glm(Y ~ X1 + X2, family="binomial", data=dd1)

# second data set
dd2 <- data.frame(X1=rnorm(50), X2=rnorm(50), Z=rnorm(50))
dd2$Y <- with(dd2, rbinom(50, size=1, p=1/(1+exp(-2-X1+2*X2-Z))))

# linear predictor based on mod1
dd2$pred1 <- predict(mod1, newdata=dd2, type = "link")
# use X1-X2 based predictor as offset
mod2 <- glm(Y ~ Z+ offset(pred1), data=dd2, family="binomial")
summary(mod2)

The output is:

Call:
glm(formula = Y ~ Z + offset(pred1), family = "binomial", data = dd2)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.6432  -0.5135   0.2310   0.4945   1.5744  

Coefficients:
            Estimate Std. Error z value Pr(>|z|)  
(Intercept)   -0.205      0.440  -0.466   0.6413  
Z              1.070      0.418   2.560   0.0105 *
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 38.598  on 49  degrees of freedom
Residual deviance: 30.536  on 48  degrees of freedom
AIC: 34.536

You can't really see the presence of the offset here, but for comparison, here is the output without an offset term:

 Call:
glm(formula = Y ~ Z, family = "binomial", data = dd2)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-2.1183  -1.1191   0.5753   0.8910   1.6962  

Coefficients:
            Estimate Std. Error z value Pr(>|z|)   
(Intercept)   1.0911     0.3767   2.896  0.00377 **
Z             0.9369     0.3888   2.410  0.01597 * 
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 62.687  on 49  degrees of freedom
Residual deviance: 54.906  on 48  degrees of freedom
AIC: 58.906

The estimates of the coefficient of $Z$ happen to be pretty close to each other, but you can see that the deviance is quite different, and the residual deviance is much lower when the additional predictor is included.