Jost diversity index and the effective number of species

For an excellent introduction to diversity and a discussion of several popular metrics, see Lou Jost's web site.

Effective number of species
The effective number of species is described by Chao, Chui and Jost (2010) and is defined as follows. Suppose a given metric has value X for a given set of abundances. The effective number of species is then the number of species in a community with equal abundances that would give X for this metric.

Using an effective number of species is compelling because it has a natural interpretation and makes all metrics comparable to each other, while "raw" metrics such as entropy have no obvious connection to a number of species, and different raw metrics cannot be compared.

For example, I created OTUs for a soil community with 3,268 OTUs which had a Shannon entropy of 3.68. With 39 even abundances, the Shannon entropy is 3.66 and with 40 even abundances, the entropy is 3.39, so we can see that the effective number of species should be more than 39 and less than 40. Using Jost's formula, we find that the effective number of species for a Shannon entropy of 3.66 is 39.7. This indicates that while the community had many OTUs, there were relatively few high-abundance clusters that accounted for most of the reads.

Jost index
In the same 2010 paper, Choa, Chui and Jost describe a family of metrics based on "Hill numbers" with a single parameter (q) that determines how abundance is weighted. I call these the Jost indexes of order q. Each of these metrics gives an effective number of species. The parameter q is the power to which the species frequencies are raised. With q=0, any frequency raised to the power of zero is one and the Jost index with q=0 is exactly equivalent to richness. With q=1, the Jost index is equivalent to the Shannon entropy. Higher values of q correspond to other, less well-known metrics.