I am attempting to analyze my (experimental psych) data in SPSS, and I have a few questions regarding the kind of analysis I should be using (GEE or GLMM), how I should be interpreting the output, and how I should be selecting the best fitting model.
(Disclaimer: Regression analyses in general are fairly new to me, as my data can usually be analyzed using simple non-parametric statistics or ANOVAs).
Here is a little about my current data set: the DV is binary (a yes/no response) and I am interested in looking at the effect of several IVs (all categorical, 2 levels each) on that measure. Given that all my variables are experimentally manipulated, I believe they would be considered fixed rather than random effects. One of my IVs (condition: A, B) is a within-subjects factor, and the rest of my IVs are between-subject factors.
These are my main concerns:
1) Should I be using GLMM or GEE? I get the same results for both, in terms of which effects are significant (which is good), but the parameter estimates are obviously different. I’m leaning towards GEE, given that I don’t have any random effects. Is this appropriate given the nature of my data?
2) If I only have dichotomous variables, does the correlation structure in GEE really matter?
2) I’m not quite sure how to interpret the parameter estimates for either GEE or GLMM, particularly in the case of interaction effects. Based on what I’ve read, I have been interpreting my main effects using the exponential coefficient (this is the odds ratio, yes?). So if I set my reference category to 0 (a “no” response), then I should be interpreting the exponential coefficient as the probability of participants responding “yes” in a given condition, relative to the other condition? Ex: If Exp(beta) for condition A is 3.267, then the probability that participants will say “yes” in condition A is 3.276 times the probability that participants will say “yes” in condition B, all other things being equal? (Is this how one would usually report this kinds of result?) Even if this is accurate, I’m still not sure how to figure out what is going on with the significant 2-way interactions. (Also, since the output only gives parameter estimates for one level of each of the effects. Does this mean I should to re-run the analysis with a different reference category to have a clear picture of the nature of these effects?)
3) I’m unsure about the process of selecting the appropriate model. Is there a standardized procedure for doing this? Just by trial and error I think I have found the best model (using GEE), but I’m not sure how to describe my model selection process when I report it, or whether mine was a valid means of doing so. Basically, I ran a simple model with just main effects, then dropped all the non-significant factors. Dropping the non-significant main effects actually seemed to reduce the fit of the model (why would that be?) so I kept all the main effects in the model, and then added all the 2- and 3-way interactions to see if any interactions were significant. Then I dropped everything except the main effects and the two 2-way interactions that were significant, and compared that final model to the original main-effects-only model. The model that included the two interactions was better (i.e. had a lower QICC and QIC) than the main-effects-only model, so I stopped there. However, one of the main effects that was significant in the main-effects-only model is no longer significant in the final model. I'm not sure why that would be, or what this means.
Thank you in advance!
