I want to understand how well a device (Device 1) estimate real activity (measured by Device 2) under different conditions: different time intervals for which I calculate activity and amount of data per time interval for Device 1. Activity data comes from 6 individuals (longitudinal data). Below I show a plot where you can see the relationship among variables:
As you can see, each plot represents a different time interval (2 hours, 1 hour, 0.5 hours, 0.25 hours and 0.1 hours from upper-left to lower-right respectively). Colour intensity of the points represents the amount of data for Device 1 per time interval. Different symbols represent different individuals.
What I want is to know if the relationship between the activity of the Device 1 and the Device 2 changes among time intervals or among amount data levels. To do so, I used a linear mixed-effects model to account for non-independent structures associated with the individuals (data over time from 6 individuals). I transformed activity data from both Device 1 and Device 2 to remove/alleviate heteroscedasticity problems. I also create a variable called n.data_level which has 3 levels (High,Medium and Low) to explore the importance of the amount of data when predicting real activity using the Device 1. Below I show the structure of my data:
> str(df4)
'data.frame': 7911 obs. of 5 variables:
$ ID : Factor w/ 6 levels "HAle","HAnto",..: 1 1 1 1 1 1 1 1 1 1 ...
$ VeDBA.V13AP : num -3.9 -6.92 -7.13 -3.82 -2.98 ...
$ VeDBA.X16 : num -4.56 -4.96 -5.47 -3.59 -4.28 ...
$ n.data_level : Ord.factor w/ 3 levels "Low"<"Medium"<..: 1 2 2 1 1 1 1 2 3 3 ...
$ Time.Interval: num 2 2 2 2 2 2 2 2 2 2 ...
What I did first was to set random structure of my model:
mod1<-lmer(Activity.Device2~Activity.Device1 + Time.Interval + n.data_level + Activity.Device1:Time.Interval + Activity.Device1:n.data_level + (1|ID),data =df, control = lmerControl(optimizer ="Nelder_Mead"),REML=T)
mod2<-lmer(Activity.Device2~Activity.Device1 + Time.Interval + n.data_level + Activity.Device1:Time.Interval + Activity.Device1:n.data_level + (Activity.Device1|ID),data =df, control = lmerControl(optimizer ="Nelder_Mead"),REML=T)
mod3<-lmer(Activity.Device2~Activity.Device1 + Time.Interval + n.data_level + Activity.Device1:Time.Interval + Activity.Device1:n.data_level + (Time.Interval|ID),data =df, control = lmerControl(optimizer ="Nelder_Mead"),REML=T)
mod4<-lmer(Activity.Device2~Activity.Device1 + Time.Interval + n.data_level + Activity.Device1:Time.Interval + Activity.Device1:n.data_level + (n.data_level|ID),data =df, control = lmerControl(optimizer ="Nelder_Mead"),REML=T)
AIC(mod1,mod2,mod3,mod4)
df AIC
mod1 10 8038.888
mod2 12 7821.262
mod3 12 8040.658
mod4 15 8030.055
Then, I set the fixed structure of my model.
Note: I used (1|ID) for the random effect since using (Activity.Device1|ID) gave convergence problems.
mod1<-lmer(Activity.Device2 ~ Activity.Device1 + Time.Interval + n.data_level + (1|ID),data = df, REML = F, control = lmerControl(optimizer ="optimx", optCtrl=list(method='L-BFGS-B')))
mod2<-lmer(Activity.Device2 ~ Activity.Device1 * Time.Interval + n.data_level + (1|ID),data = df, REML = F, control = lmerControl(optimizer ="optimx", optCtrl=list(method='L-BFGS-B')))
mod3<-lmer(Activity.Device2 ~ Activity.Device1 * n.data_level + Time.Interval + (1|ID),data = df, REML = F, control = lmerControl(optimizer ="optimx", optCtrl=list(method='L-BFGS-B')))
mod4<-lmer(Activity.Device2 ~ Activity.Device1 * Time.Interval * n.data_level + (1|ID),data = df, REML = F, control = lmerControl(optimizer ="optimx", optCtrl=list(method='L-BFGS-B')))
AIC(mod1,mod2,mod3,mod4)
df AIC
mod1 7 8183.510
mod2 8 8185.504
mod3 9 7993.096
mod4 14 7916.357
My doubt comes when I see residuals versus fitted values since I see some heteroscedasticity likely associated to the fact that at small time intervals (0.1 and 0.25 hours) there is more variance than the higher ones (0.5, 1 and 2 hours).
After reading this post and this one, I was thinking of weighting as a solution in lmer, but I am not sure how I should do it. My first try was this:
mod4<-lmer(Activity.Device2~ Activity.Device1 * Time.Interval * n.data_level + (1|ID), weights = Time.Interval, data = df4, REML = F, control = lmerControl(optimizer ="optimx", optCtrl=list(method='L-BFGS-B')))
Does it make sense to just do this? I don't know if I might indicate that the source of variance is in the Time.Intervals 0.1 and 0.25.
I would appreciate any help!!
