1
$\begingroup$

dispersion around line y = x

As shown in the figure, the lines in both panels are $y = x$. The lines are not regression lines. I see that in the right panel the data points scatter further away from the $y = x$ line. Abundance.y and Abundance.x are supposed to be the same so that they should gather close around $y =x$ line. We are testing whether the left panel points are "tighter" around $y = x$ than the right panel points.

What is a good test for this?

$\endgroup$
4
  • 1
    $\begingroup$ Welcome to our site! Would it be possible to include your data, or if that is not possible for confidentiality reasons, some "example data" that has the same kind of properties? $\endgroup$
    – Silverfish
    Feb 10, 2016 at 15:41
  • $\begingroup$ Let $Z_i = (X_i - Y_i)^2$ (or any other loss function, e.g. $|X_i-Y_i|$), then your question can be recast as a question about whether the $Z$s tend to be larger in the left panel vs. the right panel; you could do various tests of means or rank based tests to investigate that question $\endgroup$
    – linksys
    Feb 10, 2016 at 15:44
  • $\begingroup$ Since, mathematically, $y=x$ is equivalent to $y-x=0$, any measure of variation of $y-x$ will do the job. But this makes some implicit assumptions about $y$ and $x$. In particular, if you are modeling them as random variables but with different variances, then you will need something a little different. Could you therefore tell us more about what these variables represent and the purpose of the test? $\endgroup$
    – whuber
    Feb 10, 2016 at 15:45
  • $\begingroup$ Why not run the regression, and test how different the slope is from 1? $\endgroup$
    – ChinG
    Feb 10, 2016 at 16:03

2 Answers 2

0
$\begingroup$

Have you heard about Bland-Altman methodology ? I assume your data is paired given the graph you make.

By computing a classical prediction interval on the data difference $X=(Abundance.y - Abundance.x)$, for each of your graph, you can simply compare which interval is the smallest. You can also see if there is a discrepancy from the reference line by looking if the difference is close to 0, parallel to the 0 horizontal reference line, or not.

Here is the prediction interval (assuming your Bland-Altman plot is "horizontally shaped": $$\overline{X}_n\pm T_a {s}_n\sqrt{1+(1/n)}$$

But please don't look at determination coefficient $R^2$ !!

$\endgroup$
4
  • $\begingroup$ Could you explain what you mean by "compare which interval is the smallest" and how it is relevant to the question about comparing two such datasets? $\endgroup$
    – whuber
    Feb 10, 2016 at 15:56
  • $\begingroup$ by looking at the width of the interval, you should be able to see which one is the smallest. Smallest mean 'data are closer', as we are looking at the differences. You can also look if the interval is centred around 0. $\endgroup$ Feb 12, 2016 at 11:12
  • $\begingroup$ That's a reasonable visual comparison. But what would you propose for the test requested in the question? $\endgroup$
    – whuber
    Feb 12, 2016 at 13:25
  • $\begingroup$ Yes, the problem is that tests are written for parameters and never for individual values (which would combine basically one test for the mean and one test for the variance). In my opinion, in the same way you can use CI to look at the results of an hypothesis testing on the difference between means, you can look at the PI to have the results of a test that would apply on the individual values (hence combining mean + sd). But there is no perfect way to write the set of hypothesis in frequentist framework, I guess (the problem is typically a Bayesian one). $\endgroup$ Feb 17, 2016 at 9:48
1
$\begingroup$

Concordance correlation measures agreement, i.e. quantifies how far $y = x$, rather than how far $y = a + bx$. The idea is most often associated with Lin, but was earlier discussed at least by Mielke and Krippendorff.

More at

Does concordance correlation require data to be normally distributed?

Does the concordance correlation coefficient make linearity or monotone assumptions?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.