I have community abundance data where each species has a mean and a standard deviation (determined by replicate samples). I am comparing the diversity of two communities. The various diversity metrics (shannon, simpson, ...) use the mean abundances to calculate diversity, but that means that the uncertainty information provided by the standard deviations is lost. Is there a way to include the standard deviations in the diversity calculation? In other words, is there a way to propagate uncertainty through the diversity metric calculation?

I know how to propagate error of sums (for $C = A + B$, then $S_C^2 = S_A^2 + S_B^2$)

And how to propagate error of mult/div (for $C = A * B$, then $(S_C/C)^2 = (S_A/A)^2 + (S_B/B)^2$

And I have some vague memories that to propagate uncertainty through more complex equations (like the diversity metrics that have exponents) I'd need to remember some calculus. This seems like a common problem though, so I'm assuming an ecologist or statistician has already figured it out. However, all I can find are ways to estimate uncertainty using bootstrapping of a single sample, not by incorporating known standard deviations. Is there an equation (or even better R package) that can do these diversity uncertainty calculations?

(note: I only have the mean $\pm$ sd, not the original abundances of each replicate. So it is not possible to get a conservative sd by repeating the diversity calculation on each replicate and calculating mean and sd of the replicate diversities.)

  • $\begingroup$ Welch two-sample t test (or if the two population variances are assumed to be equal), pooled two-sample t test. Both are for normal data. // Otherwise Wilcoxon RS test. $\endgroup$
    – BruceET
    Jul 2 at 22:11
  • $\begingroup$ @BruceET my question is how to calculate a diversity metric's variance, not how to choose significance tests. I don't know the variance of the diversity metric, only the variances of each species. Diversity metrics (like Shannon diversity) are a summary metric for all the species in the community. I only know variance for each species, I want variance of the summary metric. $\endgroup$
    – rrr
    Jul 2 at 23:47
  • $\begingroup$ @BruceET: indices of (bio)diversity is an established term, so I am quite sure he does not intend it to mean standard error. He mentioned entropy, Simpson's and there are many others. ... Can you tell is more of your data, how many indices are you calculating? $\endgroup$ Jul 3 at 20:05
  • $\begingroup$ Thanks @kjetilbhalvorsen. The data itself is microbial community data (via 16S rRNA sequencing). The structure of this type of data is very skewed, closer to log normal than normal, and also tends to be very sparse (most species are observed in only a few samples). Because of the sparsity, the choice of diversity metric can show very different results depending on how much weight it gives rare organisms. Generally, I calculate several metrics to get a better sense of what's driving the differences, but I have no clear favorite metric. $\endgroup$
    – rrr
    Jul 6 at 19:01
  • $\begingroup$ Please include new information as an edit to the post! Not everybody reads comments ... $\endgroup$ Jul 13 at 2:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.