I conducted a study with 28 subjects to measure the effect of an intervention. Thus, I have 3 data points from each subject: pre-treatment, treatment, and post-treatment.

I tried running a generalized mixed-effect linear model, since my data is binomial and non-independent. In this model, I included subject as a random effect (together with three fixed effects I wanted to look at). My problem is that the model would not converge. So I was wondering whether using a generalized linear model instead would be a good alternative, as this one does converge. I know it is not the best solution, but my knowledge of statistics is pretty limited and I cannot think of a better alternative. Even though I would not be able to account for those random effects, would this model still give me some useful information/results?

Note: I tried using different optimizers and none of them worked, since I still got convergence warnings.

More information about my study:

It's a teaching-intervention study. I looked at whether explicit instruction during a language class led to better use of two particular linguistic structures. I also had two groups (corresponding to 2 different classes: intermediate learners and advanced learners).

My code:

lm_general = lme4::glmer(TARGET~CONSTRUCTION*PHASE*TYPE_OF_SPEAKER + (1|SUBJECT) + (1|ITEM), data = my_df, family = "binomial")

My results (for fixed effects):

                         Estimate Std. Error z value Pr(>|z|)    
(Intercept)              1.0461     0.3352   3.120  0.00181 ** 
CONSTRUCTIONpassives    -2.8949     0.2050 -14.121  < 2e-16 ***
PHASEpretreatment       -0.1810     0.2321  -0.780  0.43548    
PHASEposttreatment      -0.3527     0.2300  -1.534  0.12504    
TYPE_OF_SPEAKERintermediate -2.3805     0.3903  -6.099 1.07e-09 ***
CONSTRUCTIONpassives:PHASEpretreatment  -18.0473  1319.5225  -0.014  0.98909    
CONSTRUCTIONpassives:PHASEposttreatment  -17.8300  1302.0983  -0.014  0.98907    
CONSTRUCTIONpassives:TYPE_OF_SPEAKERintermediate 2.6915     0.2687  10.018  < 2e-16 ***
PHASEpretreatment:TYPE_OF_SPEAKERintermediate -0.9765     0.3584  -2.725  0.00643 ** 
PHASEposttreatment:TYPE_OF_SPEAKERintermediate 0.6715     0.3100   2.166  0.03028 *  
CONSTRUCTIONpassives:PHASEpretreatment:TYPE_OF_SPEAKERintermediate 0.4458  1744.3590   0.000  0.99980    
CONSTRUCTIONpassives:PHASEposttreatment:TYPE_OF_SPEAKERintermediate -1.2249  1729.7634  -0.001  0.99943    

My warning: "Model failed to converge: degenerate Hessian with 1 negative eigenvalues"

  • $\begingroup$ Which convergence warnings? $\endgroup$ – Sebastian Mar 30 '20 at 23:44
  • $\begingroup$ @Sebastian Model failed to converge: degenerate Hessian with 1 negative eigenvalues $\endgroup$ – aprendiz Mar 30 '20 at 23:45
  • $\begingroup$ Can you post the head of the data, the code you used to do this, the warning, and what the summary said of the model object? And also some more details about the experiment: What are you measuring? How was it collected? Etc. $\endgroup$ – Mark White Mar 30 '20 at 23:48
  • $\begingroup$ @aprendiz ah ok then it really is not the optimizer $\endgroup$ – Sebastian Mar 30 '20 at 23:50
  • 1
    $\begingroup$ @MarkWhite I added more info $\endgroup$ – aprendiz Mar 31 '20 at 0:03

You have huge standard errors in these interaction groups:

                                      Estimate Std. Error z value Pr(>|z|)    
CONSTRUCTIONpassives:PHASEpretreatment  -18.0473  1319.5225  -0.014  0.98909    
CONSTRUCTIONpassives:PHASEposttreatment  -17.8300  1302.0983  -0.014  0.98907    

This usually happens when there is not at least one positive outcome and one negative outcome in each group (passives + pretreatment observations, and passives + posttreatment observations). It is called separation and prevents the model from converging.

One way to fix this is just to take that interaction out. You can also use something like Firth logistic regression, which will converge even when there is separation.

  • $\begingroup$ Thanks for the information @Nick, I know the standard errors are huge, so my question asked whether if I run a model with fixed effects only I would still be able to get some info out of it (I can't remove the interaction because otherwise there is no way to answer my RQs). And also, what is a Firth logistic regression? Would I able to run such a regression with my interaction and random factors? $\endgroup$ – aprendiz Mar 31 '20 at 0:40
  • $\begingroup$ The larger issue is that if you have separation, you can't really trust any of your estimates, even for other parameters. Are you sure you can't construct a simpler model with less interactions that answers your research question? If it is just about comparing outcomes among a pretreatment group and a posttreatment group, all the interactions might not be necessary. $\endgroup$ – Nick Mar 31 '20 at 0:58
  • $\begingroup$ I tried running two separate models (one for each type of speaker). Each of those models included an interaction between phase and construction. However, I got the same warning. Now, if I remove that other interaction that is still left, I don't think I might be able to answer my RQs, don't you think? $\endgroup$ – aprendiz Mar 31 '20 at 1:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.