I'm trying to assess the effectiveness of a program by comparing employee performance before the program vs. after the program. I have 4 years (2 years before vs. 2 years after) of individual-level employee performance data, aggregated at the monthly level. Employees could leave the company or transfer to another position, so most individuals do not have the full 48 (4*12) months of data, which leaves lots of missing values in the data set.
I looked at lots of posts and decided to use mixed model with repeated measures for this comparison.
According to answers to this post: Using lmer for repeated-measures linear mixed-effect model, I decided to configure my model as:
m1 <- lmer(Performance ~ Program + Month + Program:Month + (1+Month|Employee_Name), mydata)
Performance is a continuous variable, Program is binary (0/1) and Month is a date variable, e.g. 1/1/2016, 2/1/2016. My understanding of the equation is that: "Program : Month" allows the program has different performance across months; "1+Month/Employee_Name" allows every employee's performance to have a different intercept and coefficients.
One of my confusion is that Program and Month is correlated, with 2015-2016 months have the program (program = 1) while 2017-2018 months don't have the program (program = 0).
Regardless, I ran the model and got this warning:
singular fit Warning message: Some predictor variables are on very different scales: consider rescaling
I'm not sure how to scale the Date variable and I also researched the reason for singular fit on this post: Dealing with singular fit in mixed models
It explains "the random effects structure is too complex to be supported by the data, which naturally leads to the advice to remove the most complex part of the random effects structure (usually random slopes)."
I don't know how to address this singular issue and how to reconstruct my model to have a reasonable result.
UPDATES: I changed Month to a numeric variable, starting from 0, then divided it by 12 to transform it to a year variable. I then ran the mixed model with this formula: lmer_perf <- lmer (Quality ~ Flag + time + Flag:time + (1 | Employee_Name), data = df). It does not have singular fit error!!! The formula only randoms the intercept, but I wanted to randomize the slop as well. So I constructed the below formula:
lmer_quality <- lmer (Quality ~ Flag + time + Flag:time + (1 + time | Employee_Name), data = df)
but then got this warning message: Warning message: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, : Model failed to converge with max|grad| = 0.00722004 (tol = 0.002, component 1)
Here are the results and the residual plot from the first formula. The plot is still not good and I don't know how to interpret the results from the output.