2
$\begingroup$

I'm doing an analysis of how error committement in a particular task influences some numerical reports. I use linear mixed effects models from lmer package. I can formulate a model such as:

lmer(report ~ error + (1+error|id), data)

However, there are actually two kinds of errors which can influence participant's reports: error from current trial (currerror) but also error from previous trial (preverror). I'd like to see how both kinds of these errors influence participants reports. Crucially, it seems that both errors are not independent - i.e. committement of error in previous trial is related to higher chance of committing error also in current trial. In other words, my dependent variable (report) is loaded by two corelated factors. It seems to me that I have two options:

1) First is to make a model with two factors (possibly, although not necessarily, with interaction term), like:

lmer(report ~ currerror * preverror + (1 + currerror * preverror|id), data)

2) Second option is to run two separate analyses for two kinds of errors:

lmer(report ~ currerror + (1+currerror|id), data)
lmer(report ~ preverror + (1+preverror|id), data)

My question is: is this lack of independence a problem for mixed models? Should I then choose second option (two separate models) over the first? And COULD I do it given that this second option is also somewhat more justified by my experimental plan, because I was originally interested only in the effect of previous-trial error, and the effect of current error came up later and is somewhat more exploratory.

Could I make two "lines" of analyses - first investigating effects of previous error (main interest) and second investigating effects of current error? Or should I include both factors in any analysis that I'll do?

$\endgroup$
1
$\begingroup$

Lack of independence in predictor variables doesn't violate any of the assumptions of linear mixed models (or linear models, for that matter; this question isn't specific to LMMs) - the only assumption on the predictors is that they are measured without error (and even that assumption can be relaxed, depending on what you're trying to do).

There is a great deal of literature about multicollinearity in linear models and what to do about it; I disagree with a lot of it. Much of the advice on removing correlated/collinear predictors from the model has to do with the idea that correlated predictors "mess up your ability to make inferences", i.e. they increase the uncertainty/standard errors of the predictors with which they are correlated. Fundamentally (IMO), what messes up your inference is that you are missing information about which of the correlated predictors is actually acting; discarding them won't help. I think Graham (2003) gives useful advice.

In your case, I would indeed suggest running both sets of models (i.e., first report the model with both terms included, then report the results from the two models that each have one term included); your first, with the interaction, is the primary outcome, as it is more conservative/honest about your ability to distinguish the effects of these two predictors. The second is more of an exploration of what each predictor could explain on its own. Make sure to be clear about your intentions when writing up the results - don't p-hack!

I wouldn't say it is forbidden to analyze only the second set of models (one term at a time) ... but if I were a peer reviewer on the paper I would probably ask to see the interaction model in any case.

Graham, Michael H. “Confronting Multicollinearity in Ecological Multiple Regression.” Ecology 84, no. 11 (November 1, 2003): 2809–15. https://doi.org/10.1890/02-3114.

$\endgroup$
1
  • $\begingroup$ Thank you! So you're saying that dependence of predictors is not a problem for modelling, only that it may make me less certain about estimates of my factors? Does it have something to do with power? I'm not sure if by "two sets of models" you mean both options I mentioned in main post (1st: single model with 2 interacting factors, 2nd: two models with single factor in each)? Or do you mean my 2nd option only (i.e. two models with single factors)? Maybe I didn't make it very clear, but I was actually wondering if it'd acceptable to ONLY run separate models (2nd option), ignoring complex one. $\endgroup$
    – Bartosz M
    Apr 18 '18 at 17:08
0
$\begingroup$

Is id a key column? The name after | is the grouping factor. If id is unique, you are not aggregating anything.

I'd separate my dataset, use 70% of the data to train the model then test each model on the other 30% to verify how well they fit.

lmer(report ~ currerror + (1|id), data) # each ID has an intercept, random.
lmer(report ~ preverror + (1|id), data)
lmer(report ~ currerror + (1+currerror|id), data)
lmer(report ~ preverror + (1+preverror|id), data)
lmer(report ~ currerror + preverror + (1|id), data)
lmer(report ~ currerror + preverror + (1|id), data)
lmer(report ~ currerror + preverror + (1+currerror|id), data)
lmer(report ~ currerror + preverror + (1+preverror|id), data)
lmer(report ~ currerror + preverror + (1+currerror|id) + (1+preverror|id), data)
lmer(report ~ currerror + preverror + (1+currerror|id) + (1+preverror|id), data)
$\endgroup$
2
  • $\begingroup$ Id is a grouping factor - it's a number of participant, and each participant has few hundrets trials (rows). I'm already after deciding what I want to model as random effect, and I want to keep it maximual (basically, it's theory-driven). My question is whether I COULD and/or SHOULD split my analyses for two interdependent factors. I'm inclined to do so, again due to my theoretical assumptions, but I'm wondering if that's justified from the modelling standpoint. Do you suggest to do AIC/BIC on combined and separated models, and simply see which gets lower value? $\endgroup$
    – Bartosz M
    Apr 18 '18 at 9:47
  • $\begingroup$ I think this answer is useful if your main goal is prediction, but it's not a good idea for inference. $\endgroup$
    – Ben Bolker
    Apr 18 '18 at 15:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.