Does a statistical law exist for the phenomenon 'the more observations, the more outliers'? The other day, I found a strong relationship between number of observations (N_obs) and amount of outliers (N_outliers) (N = 20.000, r = .77, p <.001). My colleague pointed out that this might be just 'logical', because the chance of extreme cases increases with increasing cases.  
When I performed the analysis on the normalised data (N_Outliers / (n - N_Outliers), no relationship was found indeed.
Does anyone know how this phenomenon is called? Is there a statistical 'law' behind this?
 A: Without loss of generality, if you use $X > c$, where $c$ is some constant, as a rule to pick outliers, then you are basically saying that you assume that some proportion of the values are outliers. If $X$ has a cumulative probability distribution $F$, then $\Pr(X > c) = 1 - F(c) = p$, so you assume that $100p\%$ to be outliers. Of course in your case $c$ is estimated from the data, so we are talking about estimates of the probabilities. By definition, proportion $p$ of a larger value $N$ is bigger then proportion $p$ of a smaller value $M$: $pN > pM$ if $N > M$.
A: The answer is yes, but it depends on the underlying distribution.
Interpreting a comment by the OP, it looks like outliers are found with a boxplot-like rule: given data, set "fences" 1.5 times the IQR away from the quartiles themselves and declare any value beyond a fence to be outlying.
When the data are a random sample from a distribution $F$, we may analogously construct two fences for the distribution based on its own quartiles.  There are two cases:


*

*The expected number of outliers increases with sample size when $F$ assigns positive probability to values beyond its fences.

*The expected number of outliers decreases with sample size down to zero when $F$ assigns zero probability to values beyond its fences.
The argument (although mathematically a little delicate to develop in full generality) should be clear.  Consider case (1).  As the sample size increases, at least one of the data fences will tend to be within the corresponding distribution fence.  In such cases, because there is positive probability of obtaining data beyond the fence and the expected number of such data beyond the fence will be proportional to that probability and the sample size, the number of data outliers increases with sample size.  In case (2), eventually the data fences tend with high probability to lie within the range of the distribution and there are no outlying data values (with higher and higher probability).
Simulations illustrate these cases.

For sample sizes $n$ ranging from $4$ through $1000$, datasets were drawn 400 times independently from a Cauchy, Normal, and Uniform distribution; outliers were identified; and the mean proportion of such outliers was recorded and plotted in this figure.  The thick white horizontal lines show the limiting values for large $n$.


*

*The proportion of outliers from a Cauchy distribution increases with $n$, reaching the limiting proportion for the Cauchy distribution (equal to about $15.6\%$).  

*The proportion of outliers from a Normal distribution decreases with $n$, but settles to a nonzero limit (equal to about $0.7\%$).

*The proportion of outliers from a Uniform distribution decreases with $n$ and settles rapidly to zero.
Since the numbers of outliers are directly proportional to sample sizes, and their proportions all approach limits, we infer that the numbers of outliers increase with $n$ in the first two cases but decrease with $n$ down to $0$ in the last case.  This plot of the counts (on log-log axes) bears out this conclusion.


Simulation code follows.
#
# Return the proportion of outliers in a data set `x`.
#
outlier.proportion <- function(x) {
  quartiles <- quantile(x, c(1/4, 3/4))
  step <- 1.5 * diff(quartiles)
  fences <- quartiles + step * c(-1,1)
  outliers <- x < fences[1] | x > fences[2]
  mean(outliers)
}
#
# Return the proportion of outliers for a distribution given by its
# CDF `pf` and inverse CDF `qf`.
#
outlier.proportion.distribution <- function(pf, qf) {
  mapply(function(pf, qf) {
    quartiles <- qf(c(1/4, 3/4))
    step <- 1.5 * diff(quartiles)
    fences <- quartiles + step * c(-1,1)
    pf(fences[1]) + 1 - pf(fences[2])
  }, pf, qf)
}
#
# Simulate data, compute proportions of outliers, and return the 
# mean proportions found in the simulation.
#
# `sizes` is an array of sample sizes.
# `f` generates iid values from a distribution.
#
simulate <- function(f, sizes, n.sim=1e2) {
  sapply(sizes, function(n) mean(replicate(n.sim, outlier.proportion(f(n)))))
}
#
# Conduct the simulations.
#
n <- ceiling(10^seq(1/2, 3, by=1/2))
n.sim <- 4e2
set.seed(17)
p.uniform <- simulate(runif, n, n.sim)
p.normal <- simulate(rnorm, n, n.sim)
p.t <- simulate(function(n) rt(n, 1), n, n.sim)
#
# Plot the results.
#
X <- data.frame(n = rep(n, 3),
                Proportion = c(p.uniform, p.normal, p.t),
                Distribution = rep(c("Uniform", "Normal", "Cauchy"), each=length(n)))
Fences <- data.frame(Distribution = c("Uniform", "Normal", "Cauchy"),
                     xstart = rep(min(n), 3), xend = rep(max(n), 3),
                     Proportion = outlier.proportion.distribution(
                                    c(punif, pnorm, function(x) pt(x, 1)),
                                    c(qunif, qnorm, function(q) qt(q, 1))))
library(ggplot2)
ggplot(X, aes(n, Proportion)) +
  geom_segment(aes(x=xstart, y=Proportion, xend=xend, yend=Proportion), 
               data=Fences, size=1.5, color="White") + 
  geom_line(size=1.5, alpha=0.7) + 
  geom_point(aes(color=ordered(n)), size = 3, show.legend=FALSE) + 
  facet_wrap(~ Distribution) + 
  coord_trans(x = "log10") + 
  ylab("Mean Proportion of Outliers") + 
  theme(axis.text.x=element_text(angle=90))

ggplot(X, aes(n, log10(n*Proportion))) +
  geom_line(size=1.5, alpha=0.7) + 
  geom_point(aes(color=ordered(n)), size = 3, show.legend=FALSE) + 
  facet_wrap(~ Distribution) + 
  coord_trans(x = "log10")  + 
  ylab("log10(Mean Number of Outliers)") + 
  theme(axis.text.x=element_text(angle=90))

A: I'll provide a possible answer to this question. To the extent that your outlier selection method remains within the support of the underlying distribution, as your sample size increases the absolute count of outliers in the sample is also likely to increase. Many real world data sets and collections of scale data should follow this pattern. As whuber mentions below, mathematical exceptions arise due to some compact distributions. Assuming that your outlier selection method consistently yields a proportion p of "outliers", repeated sampling from the same distribution would generally lead to a convergence to a proportion p defined by your outlier selection rule. 
The proportion of outliers after repeated sampling would be expected to remain the same as the definition in terms of the underlying distribution that is considered to be an outlier (of course you would need an accurate distributional fit or a priori knowledge of the underlying distribution for the data to define these thresholds). You would also need an outlier selection strategy that maintained a proportion p of outliers as sample size increases. 
I might consider this from the perspective of the "natural tolerance" of the underlying distribution (defined as the quantiles that contain the "middle" 99.73% of the distribution in question). For a normal distribution this is the $\pm 3 \sigma$ quantiles (any standardized sample values less than -3 or greater than 3 would be outliers assuming a normal distribution is appropriate). 
For other distributions you would need to find the appropriate quantiles based on your assumption/fit of the data in question. For something like a gamma distribution (shape=1, scale=2) these cutoffs would be .002702 and 13.2155 (i.e. any value less than .0027 is an outlier and any value above 13.2155 is an outlier). Remember that the selection method for outliers is still heavily debated by statisticians and is independent of the question you posed about the count of outlier values as sample size increases.
As an aside to the question, the Pearson correlation coefficient involves specific assumptions regarding normality of both of the input variables. We would need more information about the data under investigation to assess whether it is the appropriate statistic to use or if other correlation/association statistics are more appropriate or other steps should be taken (ex. transforming the sample data using a transformation like the Box-Cox transformation or Johnson transformation). 
