0
$\begingroup$

I'm trying to fit different linear mixed models in the context of model selection. I'm using the StatsModel Python package in order to do that.

However, I encounter a very strange phenomenon. Say I start with a model having only an intercept mu, and say this model has a log-likelihood of 10. When I try to fit the same data using a model having mu+mu2, the log-likelihood actually decreases (e.g. to the value 5)... Now, that seems really weird to me, since the two models are nested, and the first model can be obtained from the second by setting mu2 to 0.

By playing a bit with the code, I managed to change to change the initial value of the parameters of the second model, such that they are the same as for the first model, with the new parameter (mu2) set to 0. But the model converge to the same value as before, to a inferior log-likelihood that the one which would be obtained with the initial condition...

Am I missing something here? Am I doing something wrong, or it the package StatsModel buggued?

PS: actual output of the code:

Simplest model:

        Mixed Linear Model Regression Results
=====================================================
Model:            MixedLM Dependent Variable: y      
No. Observations: 80      Method:             REML   
No. Groups:       10      Scale:              0.0434 
Min. group size:  8       Likelihood:         -3.2128
Max. group size:  8       Converged:          Yes    
Mean group size:  8.0                                
-----------------------------------------------------
           Coef. Std.Err.    z    P>|z| [0.025 0.975]
-----------------------------------------------------
Intercept  9.872    0.097 101.843 0.000  9.682 10.062
Group RE   0.089    0.226                            
=====================================================

Augmented model (two parameters added, a0 and b0, both set to 0 as initial condition, while the other parameters are set to the value from the simplest model as initial condition):

=====================================================
Model:            MixedLM Dependent Variable: y      
No. Observations: 80      Method:             REML   
No. Groups:       10      Scale:              0.0434 
Min. group size:  8       Likelihood:         -4.2446
Max. group size:  8       Converged:          Yes    
Mean group size:  8.0                                
-----------------------------------------------------
           Coef. Std.Err.    z    P>|z| [0.025 0.975]
-----------------------------------------------------
Intercept  9.872    0.096 103.078 0.000  9.685 10.060
a0         0.173    0.124   1.402 0.161 -0.069  0.416
b0         0.076    0.151   0.505 0.614 -0.220  0.373
Group RE   0.086    0.247                            
=====================================================
$\endgroup$
0
$\begingroup$

After a lot of googling, I realized that my problem was coming from using restricted maximum likelihood (REML) instead of maximum likelihood (ML) for the parameters optimization. This makes the likelihood not comparable between two models having different fixed effets.

Anyone interested, check this post.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.