We have a question about how to choose between two non-nested mixed linear models.
The two models in R:
Model1a <- lmer(Pain ~ Visit_Year * Sleep + (Visit_Year | ID), data = Final_sleep_pain, REML = FALSE) Model1b <- lmer(Sleep ~ Visit_Year * Pain + (Visit_Year | ID), data = Final_sleep_pain, REML = FALSE)
Both pain and sleep are time-varying variables. Pain refers to general pain intensity, ranging from 0 to 100, with a higher score meaning lower pain level;
Sleep: refers to sleep disturbance, ranging from 0 to 20, with a higher score meaning more severe sleep problem;
Visit year ranges from 0 to 11.
The data has 144233 observations and 70582 participants.
Based on the literature, there is no consistent direction of which symptom predicts which. We want to use the data-driven method, meaning let the data itself decide which model fits better, but we don’t know the criteria to decide which model is fitting better. We checked some values and showed the info in the following.
AIC, BIC, Loglikelihood, Marginal R square, Conditional R square values:
Model1a: 1298194, 11298253, -649091, 0.059, 0.58;
Model1b: 797038, 797097, -398513, 0.036, 0.62.
We are not sure which is the criteria we should look at. Also, we are not sure if we should use these criteria or some other information such as cross-validation maybe?
The reason we want to use the data-driven method is because for example if a patient complains both pain and sleep problem simultaneously, what kind of medicines should the doctors prescribe? If pain predict sleep, then pain medicine is enough; if sleep predicts pain, then sleep medications.