7
$\begingroup$

I am teaching a class on logistic regression with SPSS. The textbook supplies a sample data set with a binary predictor and two numeric covariates. The sample contains 1000 rows and a number of these entries have common values for both predictors. One predictor takes only 5 values, for example, and the other takes around 20 distinct values.

According to the documentation of SPSS, when this happens, SPSS treats the data as coming from subpopulations, defined through the common values. This seems to produce a different likelihood and different degrees of freedom for the AIC than what you get if you ignore subpopulations.

I ran the data set in R, using glm. Degrees of freedom were 997, AIC=508.93

On SPSS, I get 99 degrees of freedom (for goodness of fit purposes) and AIC=181.341. The coefficient estimates are the same in both applications.

To make matters worse, when I fit the model in SPSS with only 1 of the 2 predictors, the likelihood is LARGER than with the 2 predictor model: -87 for the 2 parameter model, and -47 for the 1 parameter model. The AIC is also dramatically smaller in the 1 parameter model, but everything else suggests that both predictors are significant and necessary. So much for the AIC criterion.

I jittered the data in R, and sent it back to SPSS. I then got much the same results as in R with glm, since there were no phantom "subpopulations" for SPSS to cope with.

Questions:

  1. can someone supply a reference to justify treating the data as coming from subpopulations (which they actually don't in this case) when the predictors contain common value sets?
  2. How am I supposed to teach model testing by comparing the deviance between two models, using SPSS and this data set, given what's going on?
  3. Can I make SPSS behave like R?
$\endgroup$
  • $\begingroup$ I don't have the answer, but it is good to know about this. I have a vague memory of SAS (Proc Logistic I think) giving different likelihoods and degrees of freedom if you specify the data as line-per-person (single-line syntax), or as aggregate data with counts of events/denominator for each unique set of covariates (events/trial syntax.) I may be suffering from a faulty memory, but potentially looking at what SAS does might give you (or someone else here) scope to work this out? $\endgroup$ – James Stanley Feb 3 '13 at 7:19
  • $\begingroup$ Just Google-booking Paul Allison's "Logistic Regression Using the SAS System: Theory and Application" (which I have a copy of in the office) -- pages 51-3 (all in the preview) talk about how SAS deals with the two different data layouts and the implications for calculating deviance statistics. I'm not mathematically minded enough to parse all the info there, but I hope it might have some hints that might be of use for the general issues, and that might lead to solving your practical problem! $\endgroup$ – James Stanley Feb 3 '13 at 7:29
4
$\begingroup$

Apparently you are using the NOMREG procedure. From the SPSS NOMREG help. Note that you can also use the newer GENLIN procedure to fit a logistic model. All three will give the same coefficients and standard errors but may differ in other outputs.

Binary logistic regression models can be fitted using either the Logistic Regression procedure or the Multinomial Logistic Regression procedure. Each procedure has options not available in the other. An important theoretical distinction is that the Logistic Regression procedure produces all predictions, residuals, influence statistics, and goodness-of-fit tests using data at the individual case level, regardless of how the data are entered and whether or not the number of covariate patterns is smaller than the total number of cases, while the Multinomial Logistic Regression procedure internally aggregates cases to form subpopulations with identical covariate patterns for the predictors, producing predictions, residuals, and goodness-of-fit tests based on these subpopulations. If all predictors are categorical or any continuous predictors take on only a limited number of values—so that there are several cases at each distinct covariate pattern—the subpopulation approach can produce valid goodness-of-fit tests and informative residuals, while the individual case level approach cannot.

$\endgroup$
  • $\begingroup$ (+1) Nice summary. The last few sentences -- about sub-population approach and validity of goodness of fit tests when predictors only take on a few discrete patterns -- was the message I got from reading pp51-53 of Paul Allison's notes on aggregate data in Proc Logistic. $\endgroup$ – James Stanley Feb 3 '13 at 22:55
3
$\begingroup$

I did some research after posting my question and basically figured it out. @JKP is right, basically. There is a detailed discussion of this in McCullagh and Nelder, Generalized Linear Models. The subpopulation model assumes that the number of categories remains constant as the population increases. Think of a contingency table where the rows and columns are constant but the cell counts tend to infinity. The degrees of freedom involve an adjustment for the number of cells. That seems to be what SPSS implements under the Generalized Linear Model menu.

This explains why I got incompatible results when I added a variable. The categories changed. The two-parameter model is no longer nested in the one-parameter model, as it would be when the analysis is done by case.

SPSS's Regression -> Binary Logistic menu allows you to enter variables one by one (in what it calls Blocks), and then does the analysis of deviance chi-squared tests for you. Using that menu I was able to get the same results as in R.

In my sample data set, the number of categories is accidental (a result of rounding off a potentially continuous variable), and would probably increase with increased sample size. I am not sure that the subpopulation model would be appropriate here. It's not the one done by the textbook, I might add, which seems to ignore the issue.

$\endgroup$
  • 1
    $\begingroup$ Thanks for the info -- can you provide page or chapter numbers for the McCullagh and Nelder reference please? $\endgroup$ – James Stanley Feb 3 '13 at 22:57
  • $\begingroup$ I am using the 1983 edition. The sections of interest are 4.3.2 (asymptotic theory for grouped data) and the following section 4.3.3 on ungrouped. Chapter 4 is about binary data, which is where this is. $\endgroup$ – Placidia Feb 7 '13 at 1:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.