9
$\begingroup$

For a given random variable (or a population, or a stochastic process), mathematical expectation is the answer to a question What point forecast minimizes the expected square loss?. Also, it is the optimal solution to a game Guess the next realization of a random variable (or a new draw from a population), and I will punish you by the squared distance between the value and your guess if you have linear disutility in terms of the punishment. Median is the answer to a corresponding question under absolute loss and mode is the answer under "all or nothing" loss.

Questions: Does variance and standard deviation answer any similar questions? What are they?

The motivation for this question stems from teaching basic measures of central tendency and spread. Whereas the measures of central tendency can be motivated by decision-theoretic problems above, I wonder how one could motivate the measures of spread.

$\endgroup$
3
  • 1
    $\begingroup$ Very interesting question. My initial approach would be that the "game" is qualitatively the same as what you already describe, except that the question expects (no pun intended) the answer to be about a range of values instead of one point, since spread without a point of reference is rather incomplete (if not meaningless) information. $\endgroup$ – Emil Sep 3 '18 at 13:08
  • $\begingroup$ Note that variance is itself an expectation - if $Y=(X-\mu)^2$ then $\text{Var}(X)=E(Y)$. $\endgroup$ – Glen_b Sep 3 '18 at 22:54
  • $\begingroup$ @Glen_b, you are right, and I got that (I should have included that in the question text). "Guess the difference between the next value and the expectation and I will punish you quadratically" would be the game. Is that the best there is? Does not sound very practical or very fun a game, IMHO. $\endgroup$ – Richard Hardy Sep 4 '18 at 6:12
2
$\begingroup$

If I have understood the question as intended, you have in mind a setting in which you can obtain independent realizations of any random variable $X$ with any distribution $F$ (having finite variance $\sigma^2(F)$). The "game" is determined by functions $h$ and $\mathcal L$ to be described. It consists of the following steps and rules:

  1. Your opponent ("Nature") reveals $F.$

  2. In response you produce a number $t(F),$ your "prediction."

To evaluate the outcome of the game, the following calculations are performed:

  • A sample of $n$ iid observations $\mathbf{X}=X_1, X_2, \ldots, X_n$ is drawn from $F.$

  • A predetermined function $h$ is applied to the sample, producing a number $h(\mathbf{X}),$ the "statistic."

  • The "loss function" $\mathcal{L}$ compares your "prediction" $t(F)$ to the statistic $h(\mathbf{X}),$ producing a non-negative number $\mathcal{L}(t(F), h(\mathbf{X})).$

  • The outcome of the game is the expected loss (or "risk") $$R_{(\mathcal{L}, h)}(t, F) = E(\mathcal{L}(t(F), h(\mathbf{X}))).$$

Your objective is to respond to Nature's move by specifying some $t$ that minimizes the risk.

For example, in the game with the function $h(X_1)=X_1$ and any loss of the form $\mathcal{L}(t, h) = \lambda(t-h)^2$ for some positive number $\lambda,$ your optimal move is to pick $t(F)$ to be the expectation of $F.$

The question before us is,

Do there exist $\mathcal{L}$ and $h$ for which the optimal move is to pick $t(F)$ to be the variance $\sigma^2(F)$?

This is readily answered by exhibiting the variance as an expectation. One way is to stipulate that $$h(X_1,X_2) = \frac{1}{2}(X_1-X_2)^2$$ and continue to use quadratic loss $$\mathcal{L}(t,h) = (t-h)^2.$$ Upon observing that

$$E(h(\mathbf{X})) = \sigma^2(F),$$

the example allows us to conclude that this $h$ and this $\mathcal L$ answer the question about variance.


What about the standard deviation $\sigma(F)$? Again, we need only exhibit this as the expectation of a sample statistic. However, that's not possible, because even when we limit $F$ to the family of Bernoulli$(p)$ distributions we can only obtain unbiased estimators of polynomial functions of $p,$ but $\sigma(F) = \sqrt{p(1-p)}$ is not a polynomial function on the domain $p\in (0,1).$ (See For the binomial distribution, why does no unbiased estimator exist for $1/p$? for the general argument about Binomial distributions, to which this question can be reduced after averaging $h$ over all permutations of the $X_i.$)

$\endgroup$
3
  • $\begingroup$ Thank you for a clear articulation of my question and an equally clear answer. Would you also have an example of $h$ that depends on all $n$ sample points, not just two? $\endgroup$ – Richard Hardy Aug 22 '19 at 6:14
  • 2
    $\begingroup$ There's a standard way to go from $2$ to $n$: compute the statistic for all pairs and average. Indeed, that produces my characterization of covariance at stats.stackexchange.com/a/18200/919. For the formal theory of this, read about U statistics. $\endgroup$ – whuber Aug 22 '19 at 12:29
  • 1
    $\begingroup$ Thank you very much! $\endgroup$ – Richard Hardy Aug 22 '19 at 12:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.