Is there a distance metric that measures the ratio between the two rows of data?

Question

I have about one hundred groups of chemical testing results data. Within each group there are between 1 and 200 chemical testing results. Each set of results contains testing data for 5 chemicals. Within each group the testing results may be from the same original sample or they may have come from a completely different sample. My task is to try and determine which testing results could have come from the same sample.

So an example from one group looks something like this:

| Test | Chem1 | Chem2 | Chem3 | Chem4 | Chem5 |
| A    | 0.01  | 0.01  | 0.51  | 0.09  | 0.42  |
| B    | 0.09  | 0.01  | 0.51  | 0.09  | 0.42  |
| C    | 2.66  | 0.01  | 0.51  | 0.09  | 0.42  |
| D    | 2.56  | 0.01  | 0.51  | 0.09  | 0.42  |

The first approach I took is not working. I tried taking the Euclidean distance. This does not work because in the above example A and B are closer together than C and D. But A and B can not be from the same sample as B is 9x A and this should not be possible given the precision of the test. I tried scaling the data but it does not address this problem.

What I think I am looking for is a distance/similarity metric that measures the ratio between the two rows of data. Is there a standard metric I can apply in this case?

Or is there an entirely different approach I should consider?

Answer 1

I agree with @Anony-Mousse. Using anything in statistics, simply because it is the default will never be the best way to go. Take your situation as an example: At the core of Euclidean distance is the difference between two values.
$$ D_E(x_i,\ x_{i'}) = \sqrt{\sum_j (x_{ij}-x_{i'j})^2} $$ That means Euclidean distance assumes the relationships are fundamentally additive in nature. Using your data:
\begin{align} &.09-.01 &<& &2.66-2.56 \\ &\qquad\ \ .08 &<& &.10 \end{align} Your comment that "B is 9x A" implies that the theoretical background in chemistry suggests the relationship should be multiplicative instead:
\begin{align} &\frac{.09}{.01} &>& &\frac{2.66}{2.56} \\[5 pt] &\quad 9 &>& &1.04 \end{align} So using ratios instead of differences will be appropriate here. Using ratios directly is not recommended, though, because ratios $<1$ scale differently than ratios $>1$ (i.e., $1/2 = .5$, but $2/1 = 2$). We can make the ratios symmetrical by taking the log of the ratios, but taking the log of a ratio is mathematically equivalent to taking the difference of two logs. Thus, if you preprocess your data by taking the log of every variable, you should be able to use Euclidean distance on the transformed data.

If I understand your situation correctly, this will give you the results you expect.

Answer 2

Don't approach this from math/statistics.

Approach this from chemistry.

What constitutes a similar sample from a chemistry point of view?

Once you've solved the chemical problem, then you can try to map this into mathematics. Either by using a customized distance, or by preprocessing the data into a format sich that e.g. Euclidean distance captures your intentions.

Answer 3

Yes there is a standard measure. It is called the Pearson correlation coefficient. It is available in scientific software such as R and scipy but it is also straightforward to compute yourself.

Answer 4

From

Cha, Sung-Hyuk. "Comprehensive survey on distance/similarity measures between probability density functions." City 1.2 (2007): 1.

You can read the reference for a technical explanation, but, intuitively, some things to point out:

• You (probably) want to be using an actual distance metric (https://en.wikipedia.org/wiki/Metric_(mathematics))
• You need a way to handle 0 values or 0 differences, which could get dangerous if you just start dividing things based on an intuition
• As-shown in the picture, these won't actually work when both values are 0, but you can just define that the distance between two 0 values is 0
• Squaring the differences will give more weight to larger ones (compared to taking the absolute value of the differences)