1
$\begingroup$

Recently, I asked a question about what procedure to use to analyse mixed data with dichotomous outcomes, see [here][1]. Now I started running some first analyses (mainly with SPSS, but I'll post the model in R code) with GLMM.

That is how I defined a model in R:

model <- glmer(errorrate ~ item + warning + item*warning + (1|ID), data=data, 
               family=binomial)
  • AV --> errorrate: dichotomous outcome variable 0 = correct, 1 = incorrect
  • UV1 --> item: item A and item B, manipulated within, 0 = item A, 1 = item B
  • UV2 --> warning: yes or no, manipulated between, 0 = no warning, 1 = warning
  • ID --> subject id

My question is the following: If the regression weight for warning is significant, how can I interpret that?

This question arose because I noticed that the results depend on how categories (i.e. UV1 and UV2) are coded. As far as I understand, this is due to the fact that those categories coded with 0 are the reference categories. Hence, if item A = 0, a significant warning effect refers only to item A. If item B = 0, a significant warning effect refers only to item B. My impression is that these effects cannot be interpreted as main effects. Rather, they remind me of some kind of a partial test of an interaction. Hence, how can these fixed effects be interpreted meaningfully when they depend on the coding? Or asked otherwise: What are they good for?

I would be very interested in hearing other's opinion on my question and my assumptions formulated above.

Answer/Solution

I think the problem was the following: Because of the 0 and 1 coding, conditional main effects were calculated. These conditional main effects are indeed the same as what is tested in the interaction. To get the "main effect", the model has to be rerun without the interaction and with the two categories coded as -0.5 and 0.5 . As a consequence, 0 is the "average" effect and comes closest as an answer to my question.

As an alternative approach for getting more information on the average effect, I thought about building an index from 0-2 (how many of the two questions were answered incorrectly?; minimum = 0, maximum = 2) and running either a Mann-Whitney U-test (groups: warning yes/no) or a independent t-test.

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ Just a note about your R code: You don't need to include item + warning because the term item*warning will be automatically expanded to item + warning + item:warning. $\endgroup$
    – smillig
    Dec 12, 2014 at 9:34
  • $\begingroup$ @smillig thanks for your reply. I run two models, one according to your suggestion and one as specified above. Indeed, the information criterion values are the same and the majority of the model parameters are, too. Is this a special case because both predictors can only be 0 or 1? I'm asking because I've seen many models specifying both single effects and their interactions. I added both SPSS outputs to the main question. $\endgroup$
    – grey
    Dec 12, 2014 at 11:47
  • 1
    $\begingroup$ possible duplicate of Interpreting main effect coefficient in different models $\endgroup$
    – StasK
    Dec 12, 2014 at 14:51

1 Answer 1

Reset to default
1
$\begingroup$

Your question has nothing to do with mixed models, or even with the binary outcome models, as you are asking about the fixed part of the model, and the way the main effects are coded. Everything in your model is significant, so you have to maintain the full interaction. The coefficient for item=0 in the first piece of output shows how the response changes, on average, when you switch from item=1 to item=0, holding other stuff constant, just as you would in any regression interpretation. The coefficient for [item=0]*[warning=0] shows by how much the outcome changes for that combination, from the baseline of [item=1]*[warning=1], on top of what is being described by the main effects. It is true that the values of the coefficients may be changing depending on how you code these main effects, but the significance will stay. Moreover, the estimates (and tests) for any particular contrast are invariant with respect to how the main effect are coded.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Based on the SPSS output from the "2 models", I gather that item*warning only in SPSS does not include the 'main effects', which is an erroneous & dangerous model to fit. $\endgroup$
    – gung - Reinstate Monica
    Dec 12, 2014 at 16:29
  • $\begingroup$ @StasK I thought that I had to use a mixed model because of the within subjects manipulation (every subject answered both questions). I am not specifically interested in the random effect, because there are only 2 measurements per participant. However, I reasoned that e.g. individual differences in the response bias should be accounted for somehow. Therefore, I assumed that I had to include a random intercept (which is, as far as I understand, (1|ID)). Am I mistaken? $\endgroup$
    – grey
    Dec 12, 2014 at 22:45
  • $\begingroup$ @StasK: You wrote "...the values of the coefficients may be changing depending on how you code these main effects, but the significance will stay." That is exactly how I felt it should be like. However, if item A = 0, the main effect of warning is not significant, whereas if item B = 0 it is. That is why I came to the assumption that the effect of the warning refers only to the reference category item which would explain why it is significant in the one but not in the other case. Hence, I am very confused because for me it looks like as if the "main effects" tested parts of the interaction. $\endgroup$
    – grey
    Dec 12, 2014 at 23:05
  • $\begingroup$ @gung As I commented above: The difference between "main effects" and interactions confuses me in this specific situation. There seem to bee only four possible combinations, which IMHO are covered with the intercept (1|1), the fixed effect item (0|1), the fixed effect warning (1|0), and the remaining part of the interaction(0|0). All other coefficients of the interaction are redundant. Therefore, I do not understand what difference the two approaches make (I assume there is one, I just didn't get it so far). $\endgroup$
    – grey
    Dec 12, 2014 at 23:11
  • $\begingroup$ @grey try searching the site & reading some of the existing threads. This is a FAQ that has been discussed dozens of times. You seem to have a significant interaction; don't try to interpret the main effect at all. $\endgroup$
    – gung - Reinstate Monica
    Dec 12, 2014 at 23:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.