A coworker and I are trying to analyze agreement between two measurement methods. I apologize in advance for needing some extra explanation due to the fact I'm an engineer whose statistics background is mostly geared toward the relationship between signal-to-noise ratio and bit error rates, and other analysis of random processes.

For method comparison, it is natural to create a Bland-Altman plot (and we've done so). However our data has some additional characteristics that Bland-Altman style analysis doesn't account for. Furthermore, we're trying to compare our results to an earlier study that published a correlation coefficient resulting from mixed-effect analysis, unfortunately this publication didn't say whether they were reporting Pearson correlation coefficient or Intra-class correlation (maybe there are others too?).

The characteristics of our data set are:

  • Multiple test subjects
  • Multiple observation instants for each test subject, sequentially ordered and equally spaced in time
  • The subjects are time varying, but receiving treatment so that the time dependent changes are not monotonic
  • At each observation instant, one measurement is made using each methods

A statistician here at our university warned us that a simple paired analysis wasn't appropriate because there's a subject-specific effect, and pointed us to mixed-effects analysis but couldn't help further.

I read several articles on mixed-effect analysis, but most of them are a comparison of groups, rather than a group of comparisons, if that makes sense. The information I found on intra-class correlation said it treats the measurements within the class interchangeably, and that seems suspect here.

This article uses mixed-effect analysis for method comparison, but has repeated measurements instead of a series of time-separated measurements. It also doesn't cover correlation coefficients on grouped data.

Here's what I've done so far, using R:

Load the data

data <- read.table(filename, header=TRUE, sep=",");

Convert variables to cases, adding factors (is it correct to create a factor for the encounter, since the time indicators are independent for each test subject?):

mdata <- within(melt(data, variable_name="method", id=c("subject", "time")), {
  subject <- factor(subject)
  time <- factor(interaction(subject, time))
  method <- factor(method)

Run linear mixed-effects model. I've chosen an autocorrelation structure for the random subject/time covariance matrices, because of the nice periodic measurements.

lm2 <- lme(value ~ method, random = list( ~1|subject, ~1|time ), corr = corAR1(), data = mdata)

I'd like to know whether I've assigned the right factors to fixed and random effects. Also, from my research I guess there should be a random effect on method*subject, but it shouldn't have AR(1) structure and I don't know how to give different structure to different random effects.

Finally, I did calculate a correlation coefficient, using intra-class correlation and encounter as the class to get measurements paired properly. But I don't think this is using the subject grouping, and as I said earlier, I don't feel like treating class members interchangeably is right.

r2 <- ICC1.lme(value, time, mdata)

What would you do differently? It seems like the lmer function was a bit easier to describe random effect nested groups, but I didn't find a way to control the correlation structure.

  • $\begingroup$ I would like to know if you solved this problem as I am in a similar situation. Thank you $\endgroup$
    – user313050
    Mar 2, 2021 at 13:09


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.