Bland-Altman (Tukey Mean-Difference) plot for differing scales

Question

I find that Bland-Altman plots for comparing two methods are extremely useful in assessing agreement. However, I'm curious if there is a similar method or transformation that can be used when the scales of the two methods are not identical, but they still measure the same underlying phenomenon.

For example, I'm attempting to compare the agreement between two methods that both measure protein content: quantitative Western blot and tryptophan fluorescence. Each gives a very different kind of measurement, but it's still reasonable to question how well they agree in measuring protein content.

My question: Is there a method similar to a Bland-Altman plot (also known as a Tukey Mean-Difference plot) that can handle different scales? My only thought was to use a Spearman correlation to compare the relative ordering of samples, but correlations are fraught with peril. Many thanks for any responses.

Answer 1

The problem with using correlations as a measure of agreement is that what they are really assessing is the ordering of the $X_i$ and $Y_i$ values, and their relative spacing, but not that the numbers themselves agree (cf., see my answer here: Does Spearman's $r=0.38$ indicate agreement?). On the other hand, if the numbers are incommensurate, it makes no sense to try to determine if they agree—it can't mean anything whether they do or don't. As a result, a Bland-Altman plot can't be of any value here. However, a correlation might offer some (albeit little) value.

From an exploratory point of view, I would start with a regular, old scatterplot. I might also do a simple linear regression and test for curvature in the relationship. It can often be the case that different measures are differentially sensitive at different ranges. For example, they might do equally well at measuring what you want in the middle of their range, but one does a better job of measuring lower values (whereas the other just starts to output the same low number, perhaps a limit of detection), and vice-versa for higher values. What I have in mind is that that the relationships aren't linear. Consider this stylized figure of the relationship between energy and the temperature of water:

Then imagine having temperature and something else, perhaps volume (ice begins to expand at lower temperatures), both as measures of energy.

Once / if you were satisfied that the relationship were linear, your ability to measure the amount of agreement would be limited to Pearson's product-moment correlation; Bland-Altman plots just won't work here.

Answer 2

Assuming you cannot convert both measures to a common set of units, and both measures are continuous and are roughly normally distributed, convert both to standardized scores (e.g., $z = \frac{x- \mu}{\sigma}$).

---

Added in response to @Nick: Bland-Altman plots plot the difference between two measures against the average of the two measures, so to be meaningful the two measures need to be measured on the same scale. Converting two measures that are on different scales to dimensionless standardised scores allows you to do the necessary calculations.

---

Added in response to @Nick (2):

Not sure what you are saying. Here is a workable example:

# Load packages
library(dplyr)
library(BlandAltmanLeh)

# Using the same conditions @Ashe used

## Set seed
set.seed(2063)

## Generate data
x <- seq(1, 40)
y <- 2 * x + rnorm(n = length(x), mean = 0, sd = 10)

## Put x and y into a dataframe
df <- data_frame(x = x,
                 y = y) %>%
    ## Add two new columns containing standarized values of x and y
    mutate(x_std = (x - mean(x)) / sd(x),
           y_std = (y - mean(y)) / sd(y))

## Bland-Altman plots of:
### i) raw x and y values
raw <- bland.altman.plot(group1 = df$x,
                  group2 = df$y,
                  main = 'Raw values',
                  xlab = 'Average of x and y',
                  ylab = 'Difference between x and y')

### ii) standardized x and y values
std <- bland.altman.plot(group1 = df$x_std,
                  group2 = df$y_std,
                  main = 'Standardized values',
                  xlab = 'Average of x and y',
                  ylab = 'Difference between x and y')

It achieves the same (in shape at least) result as the lm approach used by @Ashe, which is what you would expect since both methods are 'rescaling' the values.

Answer 3

I came up with a possible solution, so I will attempt to answer my own question. I would like some critical feedback from the community though.

I know the two phenomenon are related, so I make the assumption that I can calibrate one scale to the other scale. I will then compare agreement between the predicted values from one method to the experimental values of the other method. This method still cannot find bias in means (as @Jeremy pointed out, this isn't meaningful in this context), but it still might allow a comparison of the 95% limits.

Some code (in R) to compare:

library(ggplot2)
set.seed(2063)  #Dr. Cochrane

bland <- function(x, y, titl=''){
  gg.data <- data.frame(x=x, y=y, avg=(x+y)/2, diff=(x-y))
  g <- ggplot(gg.data, aes(x=avg, y=diff)) + geom_point(size=4) + theme_bw()
  g <- g + theme(text=element_text(size=24), axis.text=element_text(colour='black'))
  g <- g + labs(x='Average', y='Difference') + ggtitle(titl)
  g <- g + geom_hline(yintercept=mean(gg.data$diff), colour='chocolate', size=1)
  g <- g + geom_hline(yintercept=mean(gg.data$diff) + 1.96*sd(gg.data$diff), colour='dodgerblue3', size=1,
                      linetype='dashed')
  g <- g + geom_hline(yintercept=mean(gg.data$diff) - 1.96*sd(gg.data$diff), colour='dodgerblue3', size=1,
                      linetype='dashed')
  plot(g)

}

#Make some data
x <- seq(1,40)
y <- 2*x + rnorm(n=length(x), mean=0, sd=10)

qplot(x,y)
lm.data <- data.frame(x=x, y=y)

lm(data=lm.data, y~x)

#Bland-Altman of raw data
bland(x,y,'Raw Data')

#Bland-Altman of calibrated data
orig.df <- data.frame(x=x)
y.p <- predict(lm(data=lm.data, y~x), newdata=orig.df)

bland(y.p,y, 'Calib Data')
qplot(y.p,y)

If I try to directly compare $x$ and $y$, as expected, I get what would be very poor agreement:

However, if I "calibrate" the $x$ values to the $y$ scale using a linear model, agreement appears much better:

Some key thoughts:

1. I don't have to use a linear model. Any model that calibrates one scale to another would do nicely.
2. This is functionally equivalent to plotting the model residuals against the mean of $y$ and the $\hat{y}$ value. This is my biggest concern. I want to compare agreement between methods, but I could be simply evaluating the quality of the model. My current thinking is that these two are equivalent.
3. Given #2, by comparing the residuals of the model as a measure of agreement, the value of my comparison rests strongly on the assumption that the model used to calibrate is correct.

To bring it all together, if I have selected a reasonable model (#1) to calibrate one scale to another (#3), then I can reasonably compare the residuals of that model (#2) as a measure of agreement. In the 2nd example graph above, I would interpret this as 95% of all deviations are within ~20 points on the $y$ scale. I can then evaluate if these limits are reasonable for the two methods I'm trying to study.

As I said above, criticisms are welcome.