Does the order of explanatory variables in logistic regression change the results?

Question

I am curious about the consequences of changing the order of the explanatory variables in a binary logistic regression. In a recent series of logistic regressions I ran in SPSS, I found that changing the order of the explanatory variables ($z, y, x$ instead of $x, y, z$) resulted in different coefficient values and significance levels. Investigating further, I got the same results in R using the same orders. Clearly, shifting the predictors around matters in terms of the results—my question is how? (And yes, I checked to make sure I wasn't entering the variables stepwise.)

Answer 1

The order of the explanatory does not matter in logistic regression. The symptoms you describe, i.e. the unstable results, may hint at multicollinearity problems or problems of quasi-complete separation in your data. Do you also have large standard errors? As your independent variables are factors, you can tabulate them one against each other and inspect the relation between them. You should also tabulate the covariate patterns against your dependent variable. You can also start by fitting the model stepwise. Then you will see at which point the problem arises.

Answer 2

The order in which you enter the variables (into a single model) does not matter. The logistic regression model of a binary response, $Y$, with three predictors $X_{1},X_{2},X_{3}$ -

$$ \log \left( \frac{ P(Y=1) }{ P(Y=0) } \right) = \beta_{0} + \beta_{1} X_{1} + \beta_{2} X_{2} + \beta_{3} X_{3} $$

will give identical results to the "re-arranged" logistic regression model

$$ \log \left( \frac{ P(Y=1) }{ P(Y=0) } \right) = \alpha_{0} + \alpha_{1} X_{2} + \alpha_{2} X_{3} + \alpha_{3} X_{1} $$

with the coefficients re-arranged appropriately. Here some R code that verifies this:

x1=rnorm(100)
x2=rnorm(100)
x3=rnorm(100)
y=-.3 + .5*x1 + 1.2*x2 - .75*x3 + rlogis(100)
y=(y>0)
g1=glm(y~x1+x2+x3,family="binomial")
g2=glm(y~x2+x3+x1,family="binomial")
summary(g1)
summary(g2)

The code generates from the model

$$ \log \left( \frac{ P(Y=1) }{ P(Y=0) } \right) = -.3 + .5 X_{1} + 1.2 X_{2} - .75 X_{3} $$

The estimates for the coefficients of $X_{1}, X_{2}, X_{3}$ (and their significance) is the same regardless of what order you put the variables in. I'm not sure why you'd be finding results that disagree with this. A coding error may be the culprit.

Edit: Based on the comment discussion, it appears that perhaps you are using a categorical predictor and the variables you're including are dummy variables, indicating levels of the predictor. In that case, the order does matter because, in R, the first one entered will be chosen as the "reference" level of the variable and all comparisons are made with respect to it. So, if you change the reference level, you also change the thing the coefficients are being compared to, therefore the estimates and $p$-values can certainly change.

Answer 3

You could try the following in SPSS and see whether it sheds any light:

graph/scatter (matrix) outcome factor1 factor2 factor3.