Binary logistic regression on SPSS 20 - output seems inconsistent and illogical

Question

I have one categorical dichotomous dependent variable (yes/no - retention of newly learned words) and two independent variables - both are categorical. One independent variable is "delay between exposure and test" and has two levels (short time delay/week time delay). The other IV is "word type" and relates to the types of words children were exposed to - object label, colour label etc. I have run a binary logistic regression on my data (440 participants - 44 per condition of word type and time delay) using the "Enter" Method and defined both IVs as categorical using the categorical tab. The initial outputs - Block 0 "Variables not in the Equation", Block 1 Omnibus Tests of Model Coeffcients and the Hosmer and Lemeshow Test all suggest that the model is significant and the Block 1 Classification Table suggests there's been an 11% improvement in predicting the DV (when compared to the Block 0 classification table). Yet the Block 1 "Variables in the Equation" Table shows no main effect of either of the IVs nor an Interaction effect - and not by a long way (time delay p=0.37; word type p=0.37; Word type*Time Delay p=0.401. I don't know how to interpret this as all the examples I've seen in my book and online show at least one significant main effect when the model is significant. It also seems somewhat illogical for SPSS to report that the model is significant but that neither of the IVs are having a significant effect! Also, my data are quite clear and it seems very likely that there is a main effect of time delay. Retention of all word types except one fall from around 65% in the short delay to around 33% after one week.

I have tried changing the indicator from first to last (ie from short delay to the week delay) for the time delay variable and concerningly the results change - I thought when the IV was dichotomous it didn't matter which of the categories was used as the control group (indicator). Even more of a concern, it produces a main effect of word type rather than time delay.

I have used dummy coding and effectively treated each of my word type categories as different IVs and this produces completely different results. Again - this is a concern as I would expect very similar results since it should effectively be running the same data.

Can anyone help?

Answer 1

The Hosmer and Lemeshow test has nothing to do with assessing the importance of a variable. The fact that some variables are not in the model implies that you are using stepwise variable selection, which is invalid. Fully specify your model. You don't need to worry about coding of categorical variables when you do an overall anova that performs pooled (composite; chunk) tests combining all $k-1$ indicator variables for $k$ categories, you have the right information. I don't know SPSS but in R you can get automatic anova results with all the pooled effects.

When the predictor is dichotomous, the test statistic is unchanged (or just negated) when you change the coding. When you alter the reference cell for a polytomous predictor, all coefficients will change but the pooled $\chi^2$ test won't.