# Quantifying diversity of bird species

I’m wondering if anyone could recommend a diversity index to look at the differences in breeding bird territories between two sites. These are my data - the numbers represent the number of breeding pairs of each species:

c(4L, 3L, 14L, 1L, 4L, 11L, 2L, 1L, 2L, 7L, 1L, 5L, 7L, 3L, 1L, 2L, 1L, 1L, 4L, 1L, 1L, 4L, 2L, 4L, 3L, 3L, 1L, 1L, 1L, 14L, 3L, 14L) -> site1

c(27L, 25L, 21L, 23L, 1L, 2L, 26L, 3L, 1L, 16L, 3L, 12L, 29L, 2L, 5L, 1L, 17L, 2L, 6L, 19L, 6L, 3L, 3L, 5L, 1L, 1L, 8L, 7L, 1L, 2L, 5L, 2L, 6L, 2L, 8L, 1L, 5L, 1L, 25L, 1L, 1L, 4L, 1L, 26L, 7L, 1L, 1L, 13L, 1L, 30L, 34L, 13L, 2L) -> site2


I’ve tried Simpson’s index in R:

library(vegan)

diversity(site1, index = "simpson")
diversity(site2, index = "simpson")


But the diversity index is very similar for both sites (0.94 and 0.96 respectively) and I would say site2 is considerably more diverse than site1.

Are there any functions that are more sensitive when quantifying diversity?

Just about any general book on ecological methods has a section on diversity measures and there are indeed several dedicated monographs on diversity in ecology alone, to say nothing about related literatures. Ecologists can get quite agitated and even angry about which measures are good and especially about if any is best. Much of the literature is naive mathematically or scientifically, although as an outsider I won't give precise references, beyond hinting that some popular accounts are shockingly muddled.

You are too pessimistic about Simpson's measure which shows in your case for both sites a high probability of random pairs of individuals being different species. However, small differences can be important. I suggest that the Simpson measure is usually better thought of on a different scale.

You are evidently using Simpson's measure as the complement of $$R := \sum_{i=1}^S p_i^2$$, namely $$1 - R$$ or $$1 - \sum_{i=1}^S p_i^2$$. Here the notation is that there are $$S$$ species and the typical species occurs with probability $$p_i$$. The $$R$$ notation is suggested by the term repeat rate, one of many terms in the jungle.

A reference case is $$S$$ species occurring equally frequently, for which $$R = \sum_{i=1}^S p_i^2 = \sum_{i=1}^S (1/S)^2 = S/S^2 = 1/S$$. This motivates $$1/R$$ as a numbers equivalent, namely an equivalent number of equally common species. I get that measure to be about 16.1 and 24.0 for your two sites, which is a fair contrast.

Similarly, consider Shannon entropy $$H := \sum_{i=1}^S p_i \ln(1/p_i)$$ which for $$S$$ equally common species is $$S (1/S) \ln S = \ln S$$. Hence $$\exp(H)$$ is a numbers equivalent. For your sites that gives 21.6 and 30.8.

Simplest of all is just the number of species present, $$S$$ itself, which for your sites are 32 and 53.

In general, $$(1/R) \le \exp(H) \le S$$. The measure of those three that responds best to the occurrence of rare or unusual species is number of species!

These measures are all part of a family $$\sum p_i^a (\ln 1/p_i)^b$$. For Simpson's measure as a sum of squared probabilities or repeat rate $$a = 2, b = 0$$; for entropy $$a = 1, b=1$$; and for the number of species $$a=0, b=0$$. This easy unification was published by I.J. Good in 1953. There are fancier families associated with the names of C.R. Rao and M.O. Hill, but that Good family is simpler.

There are many other measures in the literature.

If you prefer $$\log_2$$ or $$\log_{10}$$ for calculating entropy, the corresponding numbers equivalent is then $$2^H$$ or $$10^H$$.

Note: This answer treats the data literally, as being say results from comparable sampling effort. The problem that the more you observe, the more you will find, and other more biological points, are well covered in other answers.

All that said, any idea that a single measure can capture all the information here is likely to prove disappointing. But that is true of any summary statistic.

• +1 Re: arguments about "which… and if any is best": Shrader-Frechette's Method in Ecology provides a phil sci take on how values and methods relate, and Grimm, V., & Wissel, C. (1997). Babel, or the ecological stability discussions: An inventory and analysis of terminology and a guide for avoiding confusion. Oecologia, 109(3), 323–334 gives an illuminating insight into how different ontological commitments translate into different operational meanings in ecology (albeit of 'stability' not 'diversity') all sharing/competing for the same term/word. Just some food for thought. Commented Jan 30, 2023 at 19:56
• Fashions come and go. A newish book by Leinster is vigorous on behalf of entropy. Commented Jan 30, 2023 at 19:59
• Nothing is simpler than the number of species observed (setting aside measurement and identification errors and debates on what is a species). Commented Jan 30, 2023 at 20:50
• +1 "Nothing is simpler than the number of species observed (setting aside measurement and identification errors and debates on what is a species [and the fact that the discrete concept "species" becomes a blurry watercolor when data are collected over evolutionary time spans])." :) Commented Jan 30, 2023 at 22:22
• @Alexis Just dropping a thought here. It would be interesting to consider a phylogeny-aware diversity measurement. I'm sure such a thing is out there but I have yet to look. Commented Jan 30, 2023 at 22:46

Firstly, I assume that you are talking about alpha-diversity (although the existence of multiple sites where the data were taken suggests that beta-diversity could be also relevant).

The simplest diversity indices (but see also here - Wikipedia has two complementary articles on the subject) are:

• Species richness : the number of species present, $$R$$.
• Shannon entropy : $$H=-\sum_{i=1}^R p_i\log_2 p_i$$
• Simpson index : $$\lambda=\sum_{i=1}^R p_i^2$$

where $$p_i$$ is the proportional abundance of species of type $$i$$ (i.e., the fraction of these species in respect to the size of the sample).

Shannon entropy and Simpson index differ from simple richness in that they give less weight to rare species, and thus less subject to fluctuations from site to site and less dependent on the sampling depth (total number of individuals sampled).

More general measures are Hill numbers $$^qD=\left(\sum_{i=1}^Rp_i^q\right)^{\frac{1}{q-1}},$$ which for $$q=0,1,2$$ are related to the diversity indices mentioned above. Hill numbers with $$q>0$$ can be interpreted as effective number of species.

As a crash course about quantifying alpha-diversity, I could recommend Measuring and Estimating Species Richness, Species Diversity, and Biotic Similarity from Sampling Data by Gotelli and Chao. This also contains information on estimating the true diversity, as well as such important tools as rarefaction curves.

Finally, relative species abundance, presented by Fisher plot or Whittaker plot can be also of interest here.

Remark: As I am more a python person than an R person, I could recommend using scikit-bio, which contains library skbio.diversity, with functions for calculating alpha- and beta-diversity. Note however that most alpha-diversity metrics are easily implemented by hand.

• Those comparing my answer will note that I use $S$ for number of species. Although notation in biology is far from unified, I don't think that $R$ is commonly, if ever, used with that meaning. The bottom line here is important. All these measures are easily calculated in a few lines in any decent language. Commented Jan 31, 2023 at 9:28
• Standardizing what symbols conventionally mean is useful. It saves people time, attention, energy and confusion. However I also think it is fine to assign variables as an author likes provided that the notation is clearly demarcated. Commented Jan 31, 2023 at 15:08
• @Galen In my defense I could say that my field is microbiote, so $R$ (Richness) is more meaningful than $S$ (Species). Richness may refer to gene richness, to species richness, to richness of mOTUs/pan-genomes/metagenomic species - all of which can be studied using the same statistical framework. Commented Jan 31, 2023 at 15:26
• You are not being criticised by me for notation choices. Clarity is top priority and consistency is good whenever possible. Thanks also for teaching me what I did not know, that $R$ is standard in microbiology. Indeed, birds are just about the opposite from your organisms in that they are big enough and attractive enough for ornithologists to be very confident of having a very complete species list (but still very excited at any new species). The idea of countable individuals of a species can be moot for corals and mosses too! Commented Jan 31, 2023 at 16:24

The other thing that's worth taking into account (although it involves similar deep rabbit holes as the other aspects of diversity metrics mentioned in other answers and comments) is that diversity measures with low $$q$$ (as in @RogerVadim's answer), i.e. richness and similar metrics, are sensitive to sample size. There are 126 total breeding pairs at site 1 vs. 468 at site 2.

Here are the rarefaction curves mentioned by @RogerVadim:

library(vegan)
## site1 and site2 from OP example above
r1 <- rarecurve(rbind(site1), tidy = TRUE)
r2 <- rarecurve(rbind(site2), tidy = TRUE)
## plot site 2 first because larger sample
plot(Species~Sample, data = r2, type = "l")
with(r1, lines(Sample, Species, col = 2))


There are 32 species at site 1; if there were only 126 breeding pairs at site 2, we would expect to see only 34 or 35 species rather than the 53 actually observed.

(You can do these computations and draw the plot slightly more elegantly if your data are organized as a species-by-site matrix with zeros filled in for species that are not observed in any samples at a particular site.)

subset(r2, Sample == max(r1\$Sample))


There are various ways to deal with issue; here I'm just pointing out a possible source of confounding/confusion.

• I think rarecurve is from the vegan package, isn't it? (To help anyone trying to do the same plot)
– llrs
Commented Feb 1, 2023 at 14:29
• Yes. I had assumed that since OP had it in their example, but I'll add it just to be explicit. Commented Feb 1, 2023 at 15:50