What are the simplest examples of nonlinear statistical functionals?

Question

I am reading Wasserman's book "All of Statistics" in which he defines a statistical functional as any function $T(F)$ of the cumulative distribution function $F(x)$ that outputs a real number. Then he goes on to define a 'linear statistical functional' as a functional $T$ for which the following condition holds:

$$T(aF+bG) = aT(F) + bT(G)$$

where $F$ and $G$ are CDFs, and $a, b$ are constants. Obviously the functionals like the mean and variance are linear. What are some examples of "nonlinear statistical functionals"?

Answer 1

• Variances. Per the Wikipedia page on mixture distributions, expectations are linear, but variances are not. (When you think about it, this is kind of obvious, because expectations involve integrating over a function, which is linear, but variances involve integrating over the square of a function, which isn't.)

  Specifically, for $n$ mixture components with means $\mu_i$, variances $\sigma_i^2$ and mixture weights $w_i$ summing to one, we have a mixture mean and variance of $$ \begin{align*} \mu =& \sum_{i=1}^n w_i\mu_i \\ \sigma^2 =& \sum_{i=1}^n w_i(\sigma_i^2+\mu_i^2-\mu^2). \end{align*}$$ The expression for $\sigma^2$ is quite different from $\sum w_i\sigma_i^2$ because of the squared means. For instance, consider two normals $N(0,1)$ and $N(1,2)$ with weights $(0.3,0.7)$, then $$ \begin{align*} \mu =& w_1\mu_1+w_2\mu_2 = 0.7 \\ \sigma^2 =& w_1(\sigma_1^2+\mu_1^2-\mu^2)+w_2(\sigma_2^2+\mu_2^2-\mu^2) = 1.91 \neq 1.7 = w_1\sigma_1^2+w_2\sigma_2^2. \end{align*}$$

  Here is a quick R simulation for people who (like me) don't trust my math-fu:

  weights <- c(0.3,0.7)
  means <- c(0,1)
  vars <- c(1,2)
  
  index <- 2-(runif(1e7)<weights[1])
  sims <- rnorm(length(which_one),mean=means[index],sd=sqrt(vars[index]))
  
  mean(sims)
  sum(weights*means)
  
  var(sims)
  sum(weights*vars)
  

• Quantiles. For instance, your CDFs could be normal distributions with different means and variances, so $aF+bG$ would be a Gaussian mixture, and $T$ could extract any quantile. Quantiles of mixtures are not simply the weighted averages of the quantiles of the components. (See here for an argument why the median is not a linear functional for mixtures of normal distributions.)

• The maximum or minimum of distributions with bounded support. If your two CDFs are for a $U[0,1]$ and a $U[0,2]$ distribution and $a,b>0$, then the mixture will have minimum $0$ and maximum $2$, regardless of the specific values of $a$ and $b$. Yes, this is not all that different from quantiles.

• The (-1)-median, which is the functional that minimizes the expected mean absolute percentage error, and is not very well known.