# Assessing which variable is more predictive in multilevel modeling. Question about seemingly conflicting results

I am analyzing data for a psychological study using multilevel models. I have a dependent variable (DV) and two predictors (A & B). DV, A, and B are all linear. I want to assess whether A or B is the better predictor of DV (30,000 observations & 1,000 participants). A and B are themselves strongly correlated (r = .70).

When I fit a multilevel regression including both A and B as predictors, A is significant (p < .001) but B is not (p > .5). Suppose that the coefficient for A is also significantly larger than the coefficient for B.

DV ~ 1 + A + B + (1 + A + B | participant)


However, when I fit two models, one using only A as a predictor and one using only B, the latter model achieves a much better fit, even when both models are fit on the exact same data (log-likelihood B >> A; I am not mixing up positive and negative log-likelihoods, to be clear):

DV ~ 1 + A + (1 + A | participant)
DV ~ 1 + B + (1 + B | participant)


Per a comment below, when I run this analysis with an interaction term (after mean centering A and B around zero), I find that A is significant, the A*B interaction is significant, but B is insignificant. The model with an interaction term achieves the best fit and best AIC. I know what an interaction is, but I'm not sure what this says about my question about A vs. B.

DV ~ 1 + A + B + A*B + (1 + A + B + A*B | participant)


So, which is the "better predictor"? I suspect that I may be confused about what "better predictor" means, but in general, how should these types of results be interpreted?

If we want this to be more concrete, suppose this is a reaction time experiment where participants are instructed to press a button upon cue. DV corresponds to their reaction time on a given trial. The cue is given both via sound and via visual indicator. Each cue varies in intensity, where A represents the intensity of the sound cue and B represents the intensity of the visual cue. Each participant did many trials of this study with varying A and B. I am interested in assessing whether sound intensity or visual intensity has a "larger bearing" on reaction time. Which is more "meaningfully" related to reaction time?

I have encountered this several times and it has always stumped me. I would appreciate any insight into how such results should be interpreted. I don't think any conflict like this would be possible in a classic (non-multilevel) regression?

• Does "group" represent the individuals here? I think that's what you imply, but it's not completely clear. Were A and B treated as strictly linear predictors in the model? Was there any pattern to the changes in A versus B levels as they were varied? Did you evaluate interactions of A with B? How many observations, individuals, and groups were involved? Please provide that information by editing the question, as comments are easy to overlook and can be deleted. The more details you can provide, the more likely that you will get a helpful answer.
– EdM
Commented Jul 10, 2022 at 18:28
• Thanks, I updated my original post per your suggestions. Yes, "group" meant individuals/participants. A and B are linear predictors of DV, which is also linear. A and B are strongly correlated (r = .7). When I include an interaction term in the multi-predictor regression, it is significant (B remains insignificant). There were 30,000 observations and 1,000 participants. Commented Jul 10, 2022 at 18:42