Quantifying agreement between datasets from different methods of measurement

Question

I have two datasets, A and B. Both ultimately come from the same source, both have the same data attributes, but the datasets use different measurement methods to derive the attributes. We want to know that both A and B are not only correlated but in agreement.

How do I compare the results of A to B in such a way that shows that the results of data B agree with, but may be more accurate than data A?

Example:

A data result

+----------+----------+----------+
| Location | MeasureA | MeasureB |
+----------+----------+----------+
| 1        | 34.56    | 234.6    |
| 2        | 123.0    | 56.78    |
| .        | .        | .        |
| .        | .        | .        |
| 256      | 68       | 453.12   |
+----------+----------+----------+

B data result

+----------+----------+----------+
| Location | MeasureA | MeasureB |
+----------+----------+----------+
| 1        | 35.1     | 234.4    |
| 2        | 122.7    | 56.7     |
| .        | .        | .        |
| .        | .        | .        |
| 256      | 68.3     | 453.14   |
+----------+----------+----------+

The main goal is to be able to define data pipeline regressions across differing methods without the results being exactly equivalent.

Answer 1

The Bland Altman plot is a much more nuanced tool for examining agreement between different methods. It allows you to examine bias as well as agreement. You can understand if the agreement is uniform across the range of values or if the methods disagree more across different parts of the range.

Determining which method is more accurate is separate step, you need to compare the two methods against a reference. I note both measures differ, do you have independent information on what the answer should be? In the absence of a reference that defines the expected truth you can't assess accuracy in a way that can say 2 is more accurate than 1.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4470095/

Answer 2

One simple way to see the the agreement between the two versions is to use the Spearman rank Correlation.

The Spearman's rank correlation coefficient is a nonparametric measure of rank correlation. It assesses how well the relationship between two variables can be described using a monotonic function.