Questions tagged [diversity]

measures of diversity (or uniformity or inequality) such as: biodiversity in ecology, income inequality in economic. Examples are entropy and simpson's indices.

Filter by
Sorted by
Tagged with
1
vote
1answer
30 views

Build tree to calculate beta diversity

I am trying to calculate beta diversity using the UniFrac distance on my data, each sample having OTUs values. To calculate it, I saw it required building a tree with percentages that represented the ...
4
votes
1answer
61 views

Clustering on n features while maximizing the heterogeneity on m remaining features

We have a random vector $X\sim p(X)$, and a set of realizations of the random vector $S=\{X_i\}_{i=1}^N$. The random vector has $n$ continuous and $m$ categorical features. I want to cluster $S$ so ...
0
votes
0answers
15 views

Continuous (not categorical) measures of diversity?

Which measures of diversity are used where? When I search for diversity indicators I find the Simpson and Shannon index, but these are both for categorical data. If I have data with some structure (e....
0
votes
0answers
28 views

Measuring entropy/concentration in multivariate datasets

I am interested in measuring how concentrated or widely dispersed a certain binary property is among a population, which is defined by a number of categorical variables. For instance, let's say I have ...
0
votes
0answers
12 views

Statistical Test for Trends in Samples from a Possibly Changing Distribution

I have some data of quantitative measurements of the language in books from the years 1800 to 2000. For each year I have between 0 and 50 samples, mostly about 10 samples per year. If I bin the data ...
0
votes
1answer
25 views

Remove points, but preserve diversity

Given a finite set of points in $\mathbb{R}^n$, all at finite distances from each other I would like to remove a certain proportion of points, in such a way as to penalise clustering. For example for $...
0
votes
0answers
29 views

How to compare community diversity of longitudinal and independent samples?

We are wanting to compare the diversity of biofouling species on longitudinal and independent samples. The first set of samples is repeated measures data, as we measured percent cover of species on ...
0
votes
0answers
18 views

How to choose which models to include in an ensemble

I have a variety of binary classification models (SVM, Logistic regression, LSTM, CNN, naive bayes etc.) which i want to ensemble to create a more robust model. What theories are useful here? What ...
3
votes
1answer
55 views

Distance matrix time series analysis? (Ecology/diversity)

I am trying to analyze a time series of ecological data. Each time point in the series is a matrix of animals-by-foods (that they were observed to eat). For each of these time points, I compute the ...
2
votes
0answers
168 views

How should I deal with spatial autocorrelation in beta GLMM (glmmTMB)? (Bird diversity)

I am trying to compare certain a taxonomic diversity index (for bird communities) calculated for squares in a map grid system (a total of 34 squares) for two different years: 1998 and 2018. My ...
3
votes
0answers
54 views

A reference request for a population statistic---experienced population density?

We all know that countries (for example) have population densities, but suppose we want to know what is the actual experienced population density (that is, if, say, in Sweden 95% of the population ...
3
votes
0answers
146 views

Which test should I use to compare the effect of a parasite infection on gut microbiome across time serie data point

I am interested in the effect of an intestinal parasite infection on the rat gut bacterial community. I have two groups of rats ("infected" and "non-infected") each consists of 6 ...
1
vote
2answers
88 views

How to include the observed values, not just their probabilities, in information entropy?

Shannon entropy measures the unpredictability in a random variable's outcome as the weighted average of the probabilities of that variable's outcomes or observed values. However, it discards the ...
0
votes
0answers
111 views

PCoA/MDS on sqrt transformed Bray-Curtis dissmilarity using cmdscale

I would like to use PCoA to visualize beta diversity of a large community. I chose Bray-Curtis as distance measure to calculate the dissimilarity matrix and would like to apply a square root ...
1
vote
0answers
55 views

Test homogeneity of multivariate dispersion with regard to the Modified Raup-Crick Metric

Please find a citation for the modified Raup-Crick Metric below Modified Raup-Crick Metric (Chase et al. 2011 As an extremely brief review, this metric generates a dissimilarity matrix containing ...
1
vote
1answer
35 views

Is there a minimum number for the calculation of the Herfindahl index?

Is there a minimum number of observations in order to assess concentration or not?
1
vote
1answer
40 views

What are the ways to normalize the vegetation data when the plot sizes of samples are different?

I have collected data on mangrove tree species density and composition from two habitat categories (sample groups) by using belt transects of various lengths between 100 x 10 M to 500 x 10 m. (3 & ...
0
votes
0answers
28 views

Diversity Measure for Imbalanced data?

In Concept Drift Adaptation by Exploiting Historical Knowledge they use Yule's Q-statistics to compute diversity between a collection of predictions. $$ \operatorname{div}(S)=1-\frac{1}{\sum_{1 \...
6
votes
4answers
320 views

How is the Herfindahl-Hirschman index different from entropy?

The Herfindahl–Hirschman Index (HHI) is a concentration measure defined as $$H = \sum_i p_i^2,$$ where $p_i$ is the market share of firm $i$. It's maximized when one firm has a monopoly and minimized ...
1
vote
1answer
54 views

Understanding the assumptions of a Poisson regression model? Modeling plant diversity

I have data on plant diversity in response to a fully crossed treatments of fertilizer and light in grassland systems: ...
0
votes
1answer
57 views

Zero Denominator in Yule's Q-statistics?

In Concept Drift Adaptation by Exploiting Historical Knowledge they use Yule's Q-statistics to compute diversity between a collection of predictions. I think the context is not really important. I ...
1
vote
0answers
15 views

Creating a fair coefficient for prize competition

I am trying to boost sales in our sales divisions by creating a small competition between them. However, the elaboration is quite tricky in order to guarantee a fair competition. There are 20 sales ...
0
votes
0answers
54 views

Is there a way calculate a gradient of dissimilarity across multiple sites using presence/absence data?

I’ve been trying to come up with a metric for species (dis)similarity for fish in a series based on presence or absence data that gives a value of 1 or 0 for each lake. From my research, it seems like ...
0
votes
1answer
83 views

What route is best for comparing diversity between 2 different sites

I am considering fungi presence and diversity in two different forest sites and looking for correlation between fungi presence/diversity and forest health. I am very new to stats, basically know ...
1
vote
1answer
94 views

Rao-Stirling diversity index

I need help to calculate the Rao-Stirling diversity index. I tried it several times but cannot achieve equal results to R packages yet (e.g. diverse). I used the Rao-Stirling diversity index as ...
2
votes
1answer
81 views

Shannon Index normalization with S distinct species per site

Given S different species and M sites . Where in every site species are unique (not repeated within the site,that is there are no 2 individuals from the same species in the same site , but can be ...
0
votes
1answer
523 views

In VEGAN ADONIS, what does "method='bray' do when I pass in a distance matrix I already have (UNIFRAC)?

Suppose I have a distance matrix created by applying the weighted unifrac measure. For instance, using a phyloseq object: ...
2
votes
2answers
73 views

Testing when the profile of a sample has changed - degrees of freedom?

I've been messing around with the Shannon index for diversity $ H' = -\sum_i^R {p_i \ln{p_i}} $, and it's associated 'evenness' measure $E = H'/H_{\text{max}}$. I'm trying to come up with a way of ...
0
votes
1answer
381 views

R: Statistically testing NMDS community differences

Situation I am comparing species communities at 6 different sites. Communities at each site were measured repeatedly over 5 years. I used NMDS plots, based on bray-curtis distance matrices using ...
4
votes
0answers
26 views

What can I use to compute a similarity (or diversity) index for a sample with “multidimensional” attributes?

Current problem: We have a batch of $n$ items for which we capture their details with $m$ attributes. It could look something like this: The goal is to compute an "index" that says how "similar" this ...
1
vote
1answer
96 views

Should I use Structural Equation Models (SEM) for my Functional Diversity (FD) study? (ECOLOGY RELATED)

I'm trying to calculate various functional diversity measures (Fric, Feve and Fdiv) for bird communities by using count data which is collected annually. My ultimate aim is to compare functional ...
1
vote
1answer
1k views

How to measure diversity (dispersion) of a categorical variable?

I have unemployment rates for 3 regions (North, Center, South). The unemployment rate is defined by (number of unemployed ($u$)/ active population ($pa$)). The unemployment rate by region ($r$) is $\...
1
vote
1answer
92 views

(Social) inequality measure based on chi-square

Is there any social inequality measure for categorical or ordinal variables, based on a goodness of fit chi squared test (distance from an uniform distribution)? ...
1
vote
0answers
124 views

Post hoc analysis survival data and maximum lifespan survival correction

Before presenting my problem I apologize that I create the same topic on the forum stackoverflow, but I think the best place for my question was here. I'm a PhD student and I'm working on survival ...
0
votes
0answers
54 views

Computing within-group diversity index in R

I am working with a df that has race and sex demographic variables. There is also a grouping variable indicating the work unit each individual observation belongs to. i.e., I want to compute a within-...
5
votes
1answer
483 views

Can you use bray-curtis distances to evaluate standardized abundances in an NMDS?

So I'm trying to run an NMDS (using the vegan package in R) for a species X site matrix. Unfortunately the abundances of species are not inherently equal (If you can use that word) meaning that not ...
3
votes
0answers
183 views

Confusion about Pareto Principle and Pareto distribution

The Pareto principle, applied to wealth for example, says that around 20% of the population holds 80% of the wealth. Accordingly, it is said that a person's wealth follows a Pareto distribution. I'm ...
0
votes
1answer
138 views

Measuring how well data is evenly dispersed

I am trying to measure how well scores from members of a group are dispersed over possible values. Think of it as a measure of diversity function. Thus, ...
1
vote
1answer
537 views

Defining Diversity in Ensemble Learning

I have a few questions regarding on how diversity is defined since I've seen differing definitions in different papers. In the paper "Measures of Diversity in Classifier Ensembles and their ...
5
votes
1answer
154 views

Why does the Shannon index take the log of number and then multiply the number by that log?

I am looking to Shannon index formula in diversity. Part of the formula I am having trouble following. For example, 50 foxes at site 1, 60 foxes site 2 and 100 foxes site 3. Across all sites there ...
0
votes
0answers
58 views

how do I calculate overyielding in ecological temporal stability

I am investigating diversity-stability relationships and I am trying to test for the portfolio effect and insurance effect with my data. I understand that overyielding indicates the portfolio effect ...
0
votes
1answer
37 views

Analysing count data for biodiversity

So I'm confused about how to analyze some data for a biodiversity survey I'm undertaking. Incidences of birds are recorded at 25 stations on four sites (so 100 points in total). My response variable ...
1
vote
1answer
19 views

Average correlation as a criterion for choosing among indices

I am modelling industrial diversity in American MSAs (Metropolitan Statistical Areas) as an inverse Hirschman-Herfindahl index (IHH). I have a choice between 4 levels of NAICS industrial ...
2
votes
0answers
60 views

Comparing biological diversity before and after

I've got counts of two species in two treatments at two timepoints. I've got six replicates of each treatment. I'd like to know how to compare treatment diversity. As it is, I've calculated the ...
0
votes
0answers
68 views

Calculating average gender diversity at companies of various sizes

We have a dataset that lists employers in a fairly unique industry, the number of employees they have and the breakdown of employees as male or female. The size range of the companies in this ...
0
votes
3answers
2k views

Understanding the output of a fixed-effects-model in R

I am researching whether female representation changes the innovation output of companies. I have panel data of 30 companies over 10 years and "diversity, sales, r&dbudget, #employees" as my ...
1
vote
1answer
459 views

Calculation of the Shannon Index plotwise or over the entire population

Imagine I inventarized a forest with 28 plots. Do I calculate the Shannon-Index for each plot and take the average, or should I calculate the Shannon-Index over the whole population? I use R and the ...
2
votes
1answer
512 views

What is statistical diversity and how is it measured? (dataset with properties of materials)

I'm curious how one would define and measure statistical diversity for a research project of mine. My motivation is to understand the statistical space of materials. Each material has a set of ...
0
votes
1answer
213 views

Statistical test for comparing species diversity based on frequency at two points

I am trying to quantify similarity between these two locations taking into account species frequency: Chicago   Blue Jay: 0.79   American Crow: 0.25   Blackpoll ...
1
vote
0answers
1k views

Small sample size - repeated measure ANOVA or something more simple?

I have data from an ecological study on wasp abundance and diversity across a forest which I am trying to analyse appropriately. We have abundance data from ten locations that were sampled twice, ...