2
$\begingroup$

Summary: Given a non-Euclidean distance matrix in which some entries represent replicate observations of an entity, is it (statistically) valid to calculate the arithmetic mean of pairwise distances across replicates?

Details: From a matrix of sample by species incidence, I have used a non-Euclidean distance metric (an index of the difference of ecological communities) to calculate a pairwise sample distance matrix. The ~70 samples represent only ~65 sites; that is, ~5 sites have been sampled multiple times, and are each represented by ~2-3 replicate observations.

I wish to sensibly collapse each set of ~2-3 replicate samples into a single observation for each site. Since the species data are binary presence/absence, taking an average at this level does not seem appropriate. I have computed the PCoA centroids for use in some analyses, but I wish to use the distance matrices themselves in other non-ordination analyses.

For this non-Euclidean distance matrix, is it valid to take the arithmetic mean (or some other average) of pairwise distances between replicates and other samples? For example, given a complete pairwise distance matrix for replicates 1A and 1B and unique samples 2 to 70, could I simply replace the two sets of rows and columns for 1A and 1B with a single row and column containing their average?

One issue I foresee is that the self-distance of replicates condensed in this way will be >0. Would it then be valid to set self-distance to 0, or would this somehow render other pairwise distances nonsensical?

$\endgroup$
5
  • 1
    $\begingroup$ Why would it be valid even to average Euclidean distances, given that they are not additive quantities? $\endgroup$
    – whuber
    Dec 28, 2016 at 16:22
  • $\begingroup$ Mmm, unconsidered assumption on my part. That would seem to answer my question firmly in the negative, and I'll stick to working with ordination centroids. Thanks. $\endgroup$
    – user119630
    Dec 29, 2016 at 1:51
  • 1
    $\begingroup$ On the contrary: the point is that since it makes perfect sense to average Euclidean distances, why shouldn't it make sense to average other kinds of distances? $\endgroup$
    – whuber
    Dec 29, 2016 at 1:52
  • 1
    $\begingroup$ Ah. I didn't realise the question was a prompt to think further; treated it as rhetorical. I think my mathematical ignorance is showing, but thank you for clarifying. $\endgroup$
    – user119630
    Dec 29, 2016 at 1:59
  • 1
    $\begingroup$ I don't think much mathematical knowledge is needed to appreciate the issues. After all, if you walk three miles to one location and one mile to another location, it likely makes sense to you that the average distance you walked is two miles. Why would such a statement be invalid? If you agree that makes sense, then what we ought to investigate next is why averaging non-Euclidean distances might be any less valid. Conceivably, for some purposes and some distances, it might not be valid--but it's hard to think of any such situation. $\endgroup$
    – whuber
    Dec 29, 2016 at 2:02

1 Answer 1

1
$\begingroup$

Answered in comments, copied below:

I don't think much mathematical knowledge is needed to appreciate the issues. After all, if you walk three miles to one location and one mile to another location, it likely makes sense to you that the average distance you walked is two miles. Why would such a statement be invalid? If you agree that makes sense, then what we ought to investigate next is why averaging non-Euclidean distances might be any less valid. Conceivably, for some purposes and some distances, it might not be valid--but it's hard to think of any such situation. – whuber

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.