I am having trouble interpreting agreement analysis in R.
I have a dataset similar to that below with multiple paired observations per subject:
x <- c(4,6,3,2,6,7,8,4,3,2,6,7,8,3,3,6,8,2,5,1,6,8,7,1,4) y <- c(7,7,3,6,3,7,7,2,4,1,3,5,0,6,3,2,1,2,8,7,3,3,4,6,3) id <- x <- c(1,1,1,1,2,2,3,3,3,3,4,4,4,4,4,4,5,6,7,8,9,9,9,10,10)
I want to assess the agreement between method
y using Bland and Altman stats.
x is the reference/gold standard method.
id represents the subjects individual id number. According to Bland and Altman publication (https://www-users.york.ac.uk/~mb55/meas/bland2007.pdf) I should examine the mean variance within subjects and between subjects using one-way ANOVA:
res.aov <- aov((x-y) ~ factor(id), data = df) summary(res.aov) Df Sum Sq Mean Sq F value Pr(>F) factor(id) 9 296.41 32.93 7.292 0.000435 *** Residuals 15 67.75 4.52 --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
I interpret this output as there is statistical difference in variance of difference between measurements using method
y within each subject. So next, I have to examine the mean bias and limits-of-agreement (LOA). First, if I ignore the subjects and calculating means and LOA for all paired observations, as if they were from different subjects I get the following:
ba <- bland.altman.stats(df$x,df$y) ba$lines lower.limit mean.diffs upper.limit -7.074781 0.560000 8.194781
When I do not ignore the subjects I first calculate the means of measurements within each subject before before calculating mean bias and LOA:
df2 <- aggregate(df[,2:3], list(df$id), mean, na.rm=TRUE, na.action=NULL) # And then calculating mean bias and LOA: ba2 <- bland.altman.stats(df2$x,df2$y) ba2$lines lower.limit mean.diffs upper.limit -5.833102 1.175000 8.183102
So, the LOA are wider if I ignore the subjects and treat the observations as different subjects. Does that mean that I should do this?
Moreover, in one parameter (not visible here) the p-value from one-way ANOVA was >0.05, but when comparing the methods mean bias was different and LOA was wider when observations were treated as different subjects compared to if I did not ignore subjects. What should I do in this case?