I am analyzing data using Liner Mixed Effects Model (LMM) with lme4 package in R. After visualizing the prediction slope and observation on a scatter plot, I realized that the estimation for the one of factor (e.g., treatment 3 in the illustration) is far from observation. after checking the data, I found that one of the nested variables (i.e. random effects) has only one observation, so that including the random intercept is adjusting the slope for the factor too much.
The following is the illustration of the very simplified research design.
If I include
participant_id as a random effect, the estimation for treatment 3 does not consider the participant C's observatio at all, which is probably cancelled out by random intercept for participant C.
res <- lmer (value ~ treatment_type + (1|participant_id))
I am aware if I don't use the mixed effect model, I am vaiolating the independent observation assumption. Because of the pusrpose of research, I really cannot just exlude treatment 3.
So, the approach I am thinking is: model excluding random intercept of
participant_it. Then, compare both models for the estimated values for coefficients and their CIs. If they are not drastically different, I would choose the
res2, the model without the random effect.
require(LmerTest) ##in order to attain p-values res2 <- lm (value ~ treatment_type) summary(res) summary(res2)
Is my approach appropriate one?
If my approach is appropriate; when including and excluding random effects generates significant differences among estimations, what should I do?
What are the other options I can use for this kind of data set?
Thank you for your help!
Detailed information about my data structure
The actual structure of my data is bit more complicated than illustrated in the image. I have data from about 40 participants nested within 15 groups (in which participants learned to-be-learned items together). I am examining the 5 types of learning conditions. Two or three of the groups engaged in more than one type of learning condition. Some participants engaged in only one types of learning condition by themselves. Some groups were also measured by the effect of learning repeatedly (i.e., immediate posttests and delayed posttests). Actually, less than half of the group only took delayed posttest. Some of the groups took more than one types of measurements, whereas the other groups just took one tyeps of measurement.
My ideal model
The following is my ideal model:
ideal_model <- lmer (value ~ learning_condition * measurement_type * test_timing + (1|group_id/participant_id)+ (1|learning_condition:participant_id)+ (1| measurement_type:participant_id)+ (1|test_timing:participant_id))
I am sorry if explanation is confusing. I can elaborate the explanation more in case I need to.