1
$\begingroup$

I'm working on a multi-level model using Mplus and trying to fit a regression at level 1 (within level).
I'm surprised to see that adding a second predictor at the within level increases my -2LL (-2*log likelihood) instead of reducing it.

A couple of information about the two models:

  • I'm only adding one L1 predictor to the initial model that contained one predictor and one DV at L1
  • The fit indices decrease a bit but remain very good (CFI > .95; RMSEA < .05; SRMR W and B < .08)
  • The -2LL for the 1st model is around 2.600 and exceeds 3.700 for the second model (with the additional predictor)
  • The initial predictor becomes marginally significant (previously significant at p < .05), while the newly added one is significant (p = .000)
  • My sample size does not change between the two models.

If it's helpful, here is the code for the second model (in which I'm adding the second predictor: Ctrl):

USEVARIABLES = ID Y X Ctrl;
WITHIN = X Ctrl;
BETWEEN = ;
CLUSTER = ID;
MISSING = ALL (-99, -9, -1);

DEFINE:
    CENTER X (GROUPMEAN);
    CENTER Ctrl (GROUPMEAN);

ANALYSIS:
    TYPE = TWOLEVEL RANDOM;
    ESTIMATOR = ML;
    ITERATIONS = 10000;
    CONVERGENCE = 0.00005;

MODEL:
    %WITHIN%
    Y ON X Ctrl; ! Predictors
    Y X Ctrl;

    %BETWEEN%
    [Y]; ! intercept
    Y;

My understanding is that -2LL should be decreasing because we are providing more explanatory variables to the model. In fact, my second predictor is significant with a higher coefficient compared to my first predictor.

Any thoughts as to why it is increasing instead is this case?
Is there something I can try to fix it?

Thank you for your help!

$\endgroup$
5
  • $\begingroup$ Given what you shared, this is strange. First question, is the outcome the same in both models? It could be a missing data problem such that the log likelihood drops from losing cases when you add in the predictor. $\endgroup$ May 17, 2021 at 16:21
  • $\begingroup$ Thank you for your comment! The DV is the same in both models as well as number of cases. I am having the same issue whether applying FIML (because I do have missing data) or using LISTWISE=ON (to use only complete data). For information, I collected my data over 4 time points for several days (3-7 days), That's why I have missing data in some points and why I intend to keep FIML (if it's not causing the issue). The first predictor is from T2, the second predictor and the DV are T4 variables. $\endgroup$
    – MEM
    May 17, 2021 at 17:03
  • $\begingroup$ I'd recommend creating a complete cases dataset for all variables in both models and running the analysis with this complete cases dataset to see if the same issue happens. Also, what exactly is your model given that you have these CFI, TLI statistics? $\endgroup$ May 17, 2021 at 22:25
  • $\begingroup$ Hello, thanks again for your feedback. I have tried using only complete data in a different dataset and the results are the same (drop in LL instead of improvement). As for the model, my aim is build up a moderation model (after this, I'm adding a third predictor supposed to be my moderator), but the LL keeps deteriorating, so... I will edit my post above to include the syntax for both models if that is helpful. $\endgroup$
    – MEM
    May 18, 2021 at 9:12
  • $\begingroup$ I would try this model in a different software for a start to see if the same problem happens. I know Mplus may be doing something unique with group mean centering so you may not be able to exactly replicate what you have here. But you could create the more usual group mean centered variables. $\endgroup$ May 18, 2021 at 13:10

1 Answer 1

1
$\begingroup$

For information, I collected my data over 4 time points for several days (3-7 days), That's why I have missing data in some points and why I intend to keep FIML (if it's not causing the issue). The first predictor is from T2, the second predictor and the DV are T4 variables.

This could be part of your problem. This model assumes that the outcome is repeatedly measured within clusters (ID). So if the outcome was only measured at T4, then a multilevel model is not valid given everything else you've said about your data (X measured at T2).

Instead, you can run this as a single level regression or path model. Assuming the data is wide (separate rows for each timepoint of a variable):

USEVARIABLES = ID Y_T4 X_T2 Ctrl;
MISSING = ALL (-99, -9, -1);

ANALYSIS:
        ESTIMATOR = MLR;
        ITERATIONS = 10000;
        CONVERGENCE = 0.00005;

MODEL:
    Y_T4 ON X_T2 Ctrl; ! Predictors
    Y_T4 X_T2 Ctrl; X_T2 WITH Ctrl; ! for full FIML specification
$\endgroup$
1
  • $\begingroup$ Hello! Thank you for your feeedback! Sorry, I was not very clear. I have collected those variables 4 times a day each day for between 3 and 7 days for each participant. That means that for each day within individuals, I have collected all my variables above, so they are at the within level (within individuals). The different time points serve to seperate measurements and estalish a stronger causal claim. However, they are not measured only once, they are repeated measures. $\endgroup$
    – MEM
    May 19, 2021 at 21:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.