# Estimating conditional effect of logistic regression

I'd like to examine the conditional effect of a new variable, say $\mathbf{Z}$, on a logistic regression previously fit to my response vector $\mathbf{Y}$ using the predictor(s) $\mathbf{X}$.

Say that the logistic regression has already been fit to the predictors $\mathbf{X}$ and that I can't/don't want to change the coefficients associated with $\mathbf{X}$. (The dataset used to fit this logistic regression was "better" in some sense than the dataset that follows, but did not contain $\mathbf{Z}$.)

I'd now like to add $\mathbf{Z}$ to my set of predictors. (Although the dataset I'm using is "worse", I still feel it's good enough to estimate the effect of $\mathbf{Z}$.) But I only want the effect of $\mathbf{Z}$ on $\mathbf{Y}$ given by current predictions based on $\mathbf{X}$.

I know that, since these models are nested, I can use the Deviance to conduct a hypothesis test of whether or not to include $\mathbf{Z}$ in the new model (under the assumption that the coefficients on $\mathbf{X}$ could be changed), but what about the actual parameter estimates?

In the linear regression case, I could simply fit a regression using $\mathbf{Z}$ to the residuals of the model based on $\mathbf{X}$ but this is problematic in the logistic regression setting, since the residuals don't exist in their usual sense.

I have been performing an optimization by minimizing the log-loss with respect to $\mathbf{Z}$ while keeping the coefficients on $\mathbf{X}$ constant. This workaround doesn't give me the standard errors I'd like, although I can repeat this procedure on subsamples of the data to get an estimate of this (assuming I have enough data).

Is there an alternative (better) way to do this?

You can include terms with fixed coefficients using an offset. Technically, an offset is a predictor with coefficient fixed at 1, so you will first need to create a new variable that has the linear combination of the $X$'s with coefficients estimated from the first model.

Model for first data set: $$logit(P(Y=1)) = \beta_1X_1 + \dots + \beta_kX_k,$$ giving estimates $\hat\beta_i$.

Model for second data set: $$logit(P(Y=1)) = \gamma Z + 1\cdot(\hat\beta_1X_1 + \dots + \hat\beta_kX_k),$$ giving estimate of $\hat\gamma$ with standard errors and all the inference.

Most software for generalized linear models (such as logistic regression) has a way to include an offset. It is most commonly used for Poisson regression, but as you can see, it can be useful for other situations as well.

Here is how this would work in R:

set.seed(456724)
# first data set
dd1 <- data.frame(X1=rnorm(50), X2=rnorm(50), Z=rnorm(50))
dd1$Y <- with(dd1, rbinom(50, size=1, p=1/(1+exp(-2-X1+2*X2-Z)))) # first model fitted with only X1 and X2 mod1 <- glm(Y ~ X1 + X2, family="binomial", data=dd1) # second data set dd2 <- data.frame(X1=rnorm(50), X2=rnorm(50), Z=rnorm(50)) dd2$Y <- with(dd2, rbinom(50, size=1, p=1/(1+exp(-2-X1+2*X2-Z))))

# linear predictor based on mod1
dd2$pred1 <- predict(mod1, newdata=dd2, type = "link") # use X1-X2 based predictor as offset mod2 <- glm(Y ~ Z+ offset(pred1), data=dd2, family="binomial") summary(mod2)  The output is: Call: glm(formula = Y ~ Z + offset(pred1), family = "binomial", data = dd2) Deviance Residuals: Min 1Q Median 3Q Max -1.6432 -0.5135 0.2310 0.4945 1.5744 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -0.205 0.440 -0.466 0.6413 Z 1.070 0.418 2.560 0.0105 * --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 (Dispersion parameter for binomial family taken to be 1) Null deviance: 38.598 on 49 degrees of freedom Residual deviance: 30.536 on 48 degrees of freedom AIC: 34.536  You can't really see the presence of the offset here, but for comparison, here is the output without an offset term:  Call: glm(formula = Y ~ Z, family = "binomial", data = dd2) Deviance Residuals: Min 1Q Median 3Q Max -2.1183 -1.1191 0.5753 0.8910 1.6962 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) 1.0911 0.3767 2.896 0.00377 ** Z 0.9369 0.3888 2.410 0.01597 * --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 (Dispersion parameter for binomial family taken to be 1) Null deviance: 62.687 on 49 degrees of freedom Residual deviance: 54.906 on 48 degrees of freedom AIC: 58.906  The estimates of the coefficient of$Z\$ happen to be pretty close to each other, but you can see that the deviance is quite different, and the residual deviance is much lower when the additional predictor is included.

• Thank you @Aniko! I hadn't thought of using an offset outside of the Poisson regression context. – Gschneider Mar 8 '15 at 2:03