How to interpret differences in explained variance (both r2m and r2c) among models that are not nested?

Question

I want to evaluate how well a device (dev.B) can predict accurately the values of another device (dev.A) that is used as a reference device. dev.B gives a value every specific time intervals (e.g. 120 seconds), and what I do is to make an averaged value per hour for the dev.B. What I want is to know if the ability of dev.B to predict values from the dev.A depends on the number of values I have per hour (some times I might have two values in one hour and others I might have 30). For the dev.A I have a unique value representative of the whole hour. Important: both devices were placed simultaneously in different animals. A fake example of the original data frame I have is below:

DateTime                 n.data      dev.A       dev.B     ID

2013-08-14 12:00:00         8        1.453       1.153      A
2013-08-14 13:00:00         3        0.653       0.834      A
2013-08-14 14:00:00        21        2.953       2.098      A
          .                 .          .           .        .
          .                 .          .           .        .
          .                 .          .           .        .

Since my purpose is to show the importance of having a large number of data from the device dev.B per hour to predict values accurately from the device dev.A, what I did is to create a categorical variable called data-quantity which classified n.data as low, medium or high, and then I run an LMM for each data-quantity. I used ID as random effect since I am interested in knowing the importance of the data-quantity in general, without considering the individual. In future studies, researchers will have only data from the dev.B for the different individuals, and thus, it is nos possible to calculate different intercepts and slopes for the different IDs. On the other hand, I think that is easier for future readers to understand the purposes of my study if I compare the explained variance of dev.A using dev.B when I have low, medium and high data-quantities instead of running one model which uses as variable of fixed factor n.data. Below I show the original data frame with the variable data-quantity:

DateTime                 n.data   data.quantity  dev.A       dev.B     ID

2013-08-14 12:00:00         8         Medium     1.453       1.153      A
2013-08-14 13:00:00         3         Low        0.653       0.834      A
2013-08-14 14:00:00        21         High       2.953       2.098      A
          .                 .           .          .           .        .
          .                 .           .          .           .        .
          .                 .           .          .           .        .

The code for the LMMs were:

mod1<-lmer(dev.A ~ dev.B + (dev.B|ID),data =data[data$data.quantity=="High",],REML=F)
mod2<-lmer(dev.A ~ dev.B + (dev.B|ID),data =data[data$data.quantity=="Medium",],REML=F)
mod3<-lmer(dev.A ~ dev.B + (dev.B|ID),data =data[data$data.quantity=="Low",],REML=F)

Relative to the explained variance, I got this:

Model   r2m      r2c

 m1    0.5518   0.9038
 m2    0.5510   0.8460
 m3    0.5168   0.6864

My doubt is about how to interpret that. Looking at r2m seems that the quantity of data per hour for the device dev.B is not critical since with low data quantity the decrease in variance explained compared to when the data quantity is high is only about 4%. However, if you look at r2c, you can see a sharper difference among data-quantities for dev.B in their ability to predict values for dev.A.

How should I interpret those results in terms of my purpose? Can I say with those results that data-quantity is important for predicting values of dev.A?

Thanks in advance.

Answer 1

Aside from the comments about running this as one model (which I agree with)--I also think you might consider reporting a different outcome altogether.

If your goal is to show prediction, then I think it would be most relevant to report the accuracy of model predictions. In other words you could report the testing/training error for each model. Then your outcome is direct a measure of what you are trying to show. Its not that r2 is wrong here, its just one step removed from the literal prediction error which it sounds like you are more interested in.