3
$\begingroup$

My lab uses what our PI calls "modified Bland–Altman plots" to analyze regression quality. I'd like to know what the generally used name is for these plots.

A Bland–Altman plot compares the difference between two measures to their average. The "modification" is that the x-axis is, instead of the average, the ground truth value. The y-axis is still the difference between the predicted and true values. In effect, the modified B–A plot can be seen as the plot of residuals from the line $y=x$—i.e. the line $predicted=truth$.

B–A plots are also called mean–difference plots, so these modified ones could be considered "truth–difference" plots, but that name is not present in literature.

Admittedly, this example has terrible bias in its prediction, shown by the deviation from the zero-line.

modified bland altman plot

Returning to the question at hand: what is the accepted name for these plots?

|cite|improve this question|||||
$\endgroup$
6
$\begingroup$

On acceptance: Acceptance of names depends on who you want them to be accepted by. Bland-Altman plots are simply Tukey mean-difference plots (and Tukey was there much earlier), so if you want statisticians to accept the name you possibly wouldn't name it after Bland and Altman. On the other hand, in some application areas (medicine or chemistry, perhaps), you'd probably get quizzical looks if you called mean-difference plots anything but Bland-Altman. [However, chances are someone was there before Tukey as well, many of these ideas are quite old.]

On a suitable name: If those things you're calling "truth" are actually "truth" (not just observations with error, say), I'd probably just call what you have a residual plot (though it looks like those differences are negative residuals); depending on how your truth was obtained you might hyphenate in a descriptive noun after "residual" (e.g. if it was based on some gold-standard calibration you might call it a residual-standard plot).

If your truth is really truth (and not just observations or even some higher-quality estimate) then you could even argue that error should be used in place of residual.

On the suitability of the plot as a diagnostic: How were those "truth" values obtained? Are those just the actual data? If so "truth" is arguably a misnomer, and in that case you'd expect there to be some negative correlation in that plot; it wouldn't necessarily suggest a problem at all. We have many threads which explain (or even prove) that plots of $y-\hat{y}$ vs $y$ will have a positive correlation, if your "difference" axis is $\hat{y}-y$ then you'd expect a negative linear trend when the regression model was appropriate.

What is the aim of the plot? How are you interpreting it?

Here's a few of the existing posts relating to the issue of plotting residuals vs data:

Trend in residuals vs dependent - but not in residuals vs fitted

Does it make sense to study plots of residuals with respect to the dependent variable?

What is the expected correlation between residual and the dependent variable?

|cite|improve this answer|||||
$\endgroup$
  • $\begingroup$ Thanks for all of this—the links suggest that I've been misinterpreting these plots, so another visualization would be superior. These "truths" are indeed observations; the scikit-learn package likes to call these y_true (as opposed to y_pred). Our hope is that there will be no correlation between predicted and "true" values. We're trying to find a good visual diagnostic of a regressor's quality. Plotting residuals against every feature is untenable, so the hope was to consolidate this into residuals against the true values. $\endgroup$ – Arya McCarthy May 23 '17 at 21:52
  • $\begingroup$ 1. You said 'Our hope is that there will be no correlation between predicted and "true" values' ... this seems an odd hope. Most people would hope for a high correlation between data and fitted values. .... 2. A common regression diagnostic for this is residuals vs fitted. (e.g. by default it's the first diagnostic produced in R when you plot regression diagnostics). Those are uncorrelated and can show a variety of problems. [Perhaps a book on regression diagnostics would be a good investment for your lab. Or even just a decent applied regression book, which should cover diagnostics.] $\endgroup$ – Glen_b -Reinstate Monica May 24 '17 at 4:32
  • $\begingroup$ Whoops—misspoke. Meant no correlation between residuals and "true" values—we want to be close to the true values always, without what looks like a bias in our estimate. $\endgroup$ – Arya McCarthy May 24 '17 at 4:33
  • 1
    $\begingroup$ "Close to true" would mean small residuals. The usual plot of residuals vs fitted should be okay for that -- if you look at standardized residuals you'd combine it with the $s$ from regression to assess their actual typical size. $\endgroup$ – Glen_b -Reinstate Monica May 24 '17 at 4:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.