4
$\begingroup$

How to correlate a set of compositions to a same-sized set of estimates of these compositions?

-> composition(estimated) vs composition(real)

Imagine you have a mixture of 5 liquids A+B+C+D+E, which in total have a concentration of 1. You have a test (or heuristic) to determine the concentration of each liquid separately but the tests are quite vague: For an evaluation of the test, you measure 100 reference-mixtures with known real concentrations [A,B,C,D and E].

Here, as an invented example, the real A-content vs the "tested" one:

enter image description here

So you can show the correlation (real vs test) for each of the 5 contents separately!

But my question is now: How to combine the 5 individual correlations? Can you recommend a way to show the correlation to the combined composition? (composition = the ratios of the individual contents A,B,C,D and E, which together are always 1)

I want to show that the tests can estimate the overall composition, not the individual concentrations, which means: composition(estimated) vs composition(real)

Considerations:

I would prefer a statistical method that can be visualized and does not just give a single number as the output. I thought of using the sum of products (estimated vs real) (AxA + BxB + CxC + DxD + ExE) and comparing it to permutations of the same contents/estimates. But I don't know how to evaluate or visualize this approach. I am thankful for any ideas or references to established methods!

$\endgroup$
  • $\begingroup$ Is there any limitation on the test that ensures that it returns values that sum to 1? In the invented example, it looks like the test returns values that are roughly double the real value on average (and much higher than that sometimes). If the same pattern holds for the other tests, the average result should sum to 2. Related, is there any correlation between the results of the tests (e.g., A and B)? If they vary together (e.g., all tests are affected by ambient light), the correlation of your tested compositon to the true value will be better than if the errors are independent. $\endgroup$ – Mark Peterson Nov 18 '19 at 14:38
  • $\begingroup$ The 5 individual tested components always add to 100%. (Both real and estimated). So the components have a negative correlation to each other. My invented example was just meant for explanatory purpose: In reality the individual correlations (A~A, B~B, ...) are quite weak with Pearson R of 0.1 to 0.5! It's my aim to look if they are more convincing when combined. $\endgroup$ – KaPy3141 Nov 18 '19 at 15:13
  • 1
    $\begingroup$ Maybe it helps: journals.plos.org/ploscompbiol/article?id=10.1371/… $\endgroup$ – marc1s Nov 24 '19 at 8:12
1
+50
$\begingroup$

It appears that you could simply run the correlation between the known composition and the result of the composition test. That is, for each component, plot the true proportion of the sample against the estimated proportion.

I will note, however, that I don't expect that the proportion will be meaningfully more accurate than the individual measures of the concentrations. The only exceptions would be if the errors are correlated (e.g., if they all run high or low together) or if one of the tests is much more accurate (because its share will be more accurately known).

Here are some sample data (in R, using tidyverse):

n_samp <- 100

sample_data <-
  tibble(
    Sample = 1:n_samp
    , Component = "A"
  ) %>%
  complete(Sample, Component = LETTERS[1:5]) %>%
  mutate(
    Real = rnorm(n(), 100, 20)
    , Tested = Real + rnorm(n(), 0, 10)
  ) %>%
  group_by(Sample) %>%
  mutate(
    Real_Prop = Real / sum(Real)
    , Tested_Prop = Tested / sum(Tested)
  ) %>%
  ungroup()

We can then calculate the correlation for each the concentration and the proportion for each component:

sample_data %>%
  group_by(Component) %>%
  summarise(
    Conc_Corr = cor(Real, Tested)
    , Prop_Corr = cor(Real_Prop, Tested_Prop)
  )

gives:

# A tibble: 5 x 3
  Component Conc_Corr Prop_Corr
  <chr>         <dbl>     <dbl>
1 A             0.899     0.896
2 B             0.918     0.896
3 C             0.905     0.912
4 D             0.896     0.908
5 E             0.895     0.901

The relationship between the concentration correlation and the proportion correlation is effectively noise. Some are slightly higher; some are slightly lower. You can play with various sizes of variability for each the concentration and the test accuracy, but this same basic reality remains.

This is different if some portion of the error is shared between the various measures. If I make half of the noise in the test result the same for each of the measures in a sample, like this:

sample_data <-
  tibble(
    Sample = 1:n_samp
    , Component = "A"
  ) %>%
  complete(Sample, Component = LETTERS[1:5]) %>%
  mutate(
    Real = rnorm(n(), 1000, 20)
  ) %>%
  group_by(Sample) %>%
  mutate(
    Tested = Real + rnorm(1, 0, 10) + rnorm(n(), 0, 10)
    , Real_Prop = Real / sum(Real)
    , Tested_Prop = Tested / sum(Tested)
  ) %>%
  ungroup()

Then, the correlations are stronger for the proportions:

# A tibble: 5 x 3
  Component Conc_Corr Prop_Corr
  <chr>         <dbl>     <dbl>
1 A             0.751     0.886
2 B             0.776     0.883
3 C             0.874     0.902
4 D             0.771     0.893
5 E             0.780     0.873

So, the benefit of this approach is entirely dependent on the relationship between the errors in testing the concentration of the various components. The stronger the (positive) relationship, the more benefit this will have.

$\endgroup$
  • $\begingroup$ I thank you for your answer and I truly respect your effort, and I up-vote you for that. But you already asked me if the estimated contents add to 1, and I confirmed. So your resealing of the composition-sum to 100% does not change anything. And even besides that, I specifically said that I already did single-component wise correlations and I am only interested in ways to combine the components to correlate the compositions as a whole. (I don't know how that would be possible) $\endgroup$ – KaPy3141 Nov 19 '19 at 12:19
  • $\begingroup$ What do you mean by "the compositions as a whole" in this context? The component-wise correlations that you described appear to be done on the concentrations without reference to the other contents. Then, you found the proportion with reference to the other contents, which is what I am addressing here (you described the ratios of the individual components as your target, though it appears that proportion addresses your question). Are you, instead, looking for a single value to describe the five-dimensional composition? $\endgroup$ – Mark Peterson Nov 19 '19 at 12:38
  • $\begingroup$ "Are you, instead, looking for a single value to describe the five-dimensional composition? " Yes, kind of like that! But neither the sum, L2, or any other norm seems to make sense, and on top of it I hate losing the information of the contribution of the 5 components to that 1 value. So I wondered if there was a way to do multi-dimensional analyses. $\endgroup$ – KaPy3141 Nov 19 '19 at 12:57
  • $\begingroup$ Something I already tried: I calculated the sum of products (expected * real): AxA + BxB + CxC + DxD + ExE, which, the higher the correlation, the bigger than 1/5. I compared the results of this observable to all permutations of products of the same compositions to determine the significance using binomial distributions. Unfortunately this failed because I get hyper-significant results in both directions. (There are biases I do not understand.) I thought maybe someone came up with a better approach. $\endgroup$ – KaPy3141 Nov 19 '19 at 13:04
  • $\begingroup$ Any single metric will, definitionally, lose information. It may be possible to do a PCA, but that will only be valuable if the variability in the composition is correlated (e.g., that you think of the components as being on a seesaw with A and B on one arm, C over the fulcrum, and D and E on the other arm; then, the only change is the angle at the fulcrum which fully describes the composition). If the composition is not correlated in that type of way (e.g., the concentration of A and E are independent of each other) then this metric will not be meaningful. $\endgroup$ – Mark Peterson Nov 19 '19 at 13:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.