# Method of collecting and comparing outliers from sets of sets of populations

Background

I am a PhD student co-supervising a Master's student in our lab. I am mostly familiar with discrete mathematics, signal processing, and programming simulations. My statistics background exists, but it is not formal or comprehensive. Without disclosing too much detail, the student's project is exploring an old biological conjecture. The idea being:

Say you have a set of populations called the long-beaks, and each population's beak length $$\sim N(\mu, \sigma)$$. Now say you look at the distribution of all population $$\mu$$'s, the statistical interpretation of the conjecture is something like, the more a $$\mu$$ is extreme in value (e.g. an outlier, but not necessarily an outlier) w.r.t. to the distribution of all $$\mu$$'s, then its $$\sigma$$ will also be extreme w.r.t. to the distribution of all population $$\sigma$$'s. In other words, if a set of populations share a common defining trait, those populations which are further from their shared central tendency are more likely to be highly variable in that trait.

Problem

There are two questions we have been continually re-visiting, and have still not settled on.

1. What is the most appropriate method of identifying those extreme values within a set of populations.
2. What is the best method of determining whether this rule holds across multiple sets of populations.

What We've Considered So Far

As for 1):

We have played with idea of using the coefficient of variation, the MADM, quartile coeffecient of dispersion, and the variation of log-transformed data, as a function of the mean or median trait values. We've experimented with arbitrary cut-offs of extremeness, and have also used methods like z-scores or the Mahalanobis distance to identify outliers (assuming $$\mu \: \& \: \sigma$$ are multivariate normal).

Is there anyone best method -- a most robust method? Or is perhaps a suite of methods more appropriate?

As for 2):

We've a hypothesis of how the data might look, should the data evidence our hypothesis. Say we min-maxed the $$\mu$$'s and $$\sigma$$'s of each set of populations, collected the 'extreme' cases from each set, and superimposed them all on the same plot. There are three plots we think that would support the hypothesis:

1. A cluster of points in the upper-left of the 0-1 space.
2. A cluster of points in the upper-right.
3. A cluster of points in the upper-right and left.

These cases all depend on the datasets collected; whether extremeness tends to be to the left and/or right of the distributions.

A plot that would not support the hypothesis is any of the last 3 cases, but where the cluster spans the y-axis and/or x-axis, i.e. there is no relationship between extreme $$\mu$$ and $$\sigma$$.

Are we perhaps overthinking this problem? Or has a common statistical test completely gone over our heads?

Comment in Answer formal to show figures:

One common method of identifying 'outliers' is to use a boxplot. Typically, boxplot outliers are greater then 1.5(IQR) above the upper quartile or smaller than 1.5(IQR) below the lower quartile (where IQR is the distance between lower and upper quartile). Ends of boxes in boxplots are distance IQR apart.

It is not unusual for truly normal samples of as many as 100 observations to show several outliers. More than half of the normal samples plotted below show boxplot outliers. Outliers are shown as black dots in the boxplots below. [It would help if you could say what importance 'outliers' may have in your particular situation.]

set.seed(2021)
n=100; m = 20
hdr = "n=100: Boxplot Outliers in 20 Samples from NORM(100,1)"
x = rnorm(m*n, 100, 1)
g = rep(1:m, each=n)
boxplot(x ~ g, col="skyblue2", pch=20, main=hdr)


If the the normal $$\sigma$$ increases (say from $$\sigma=1$$ above to $$\sigma=15$$ below), that does not increase the frequency of boxplot outliers. (Everything scales proportionately). [If you mean something different by increasing the normal standard deviation, please give details.]

set.seed(2021)
n=100; m = 20
hdr = "n=100: Boxplot Outliers in 20 Samples from NORM(100,15)"
x = rnorm(m*n, 100, 15) # change in this line only
g = rep(1:m, each=n)
boxplot(x ~ g, col="skyblue2", pch=20, main=hdr)

• If a set of populations largely share the same mean trait, then those populations have probably not had any strong selection on their shared trait. However, if there are noticeable outliers, they are the most likely to have undergone recent or ongoing selection. As a result we assume the variation in the trait is also higher. Jun 10, 2021 at 14:04
• In my sequences of 20 boxplots, notice that individual plots declare as outliers some hi/lo observations in a group that are not considered outliers in other groups.// Mechanism for declaring boxplot outliers is not much affected by outliers themselves. // Especially criteria 'CV' and 'variability of log-transformed data', may be heavily influenced by outliers themselves.// Watch out for circular reasoning. Jun 10, 2021 at 17:48
• Seems you're saying a population with 'strong selection' may me a 'mixture' of the typical distribution and another distribution, which causes greater variability and occasional outliers. Outliers alone may cause greater variability. Maybe read about 'mixture distributions'. Maybe edit your Q to explain mixing mechanism. What do you mean by "min-maxed"? Can you give examples of 1,2,3 and of proposed plots. Is your point presence of outliers alone is a good predictor of mixing? Might outliers come from experimental error instead? Do you have alternate methods to detect trait variability? Jun 10, 2021 at 17:50
• I'll try to explain the student's project a little more. The ideal dataset might be N populations, all normally distributed. The N populations are not independent of one another, they share an evolutionary history. Say N-k populations have the same mean (or nearly so) and the same standard deviation (or nearly so), where N >> k, and the remaining k populations have a mean either much lower or higher than the other N populations, and with a much larger standard deviation, regardless of having a lower or higher mean. We are looking for a way of identifying those k populations. Jun 11, 2021 at 3:30
• The student plans to collect a number of datasets and determine if this is a rule of generality. i.e. once these k outliers are identified, is it indeed true that they generally have higher variability than their conspecifics/generics? I suppose this could be done with something simple like a binomial test, but I am wondering if there is perhaps something more sophisticated? And by min-maxed I mean standardizing all conspecific/generic means and standard deviations to a 0-1 scale. Jun 11, 2021 at 3:42