4
$\begingroup$

Minimizing square loss results in predicting conditional means.

Minimizing absolute loss results in predicting conditional medians.

What loss function results in predicting conditional variances?

I have seen this answer, which works if we can assume a conditional mean of zero (which might be a safe assumption for much of my particular application), but what if we can't assume a conditional mean of zero?

$\endgroup$
3
  • $\begingroup$ In my answer to my own question I had to assume a normal distribution and a mean (or a conditional mean) of zero. This is in contrast to say squared loss which works for any distribution with a defined mean. More interestingly squared loss is not the only function that yields the mean. There are more efficient ones depending on the underlying distribution. Similarly Linex function is actually a terrible estimator for variance. stats.stackexchange.com/questions/442365/…. $\endgroup$ Aug 19, 2022 at 4:48
  • $\begingroup$ ARMA-GARCH or a similar mean-variance combo (estimating mean and variance simultaneously) estimator works much better in practice and more efficient. Almost all of those models are estimated by maximum likelihood (which is also an M-estimator by the way). Maybe you are trying to ask something else? $\endgroup$ Aug 19, 2022 at 4:55
  • 1
    $\begingroup$ Somewhat related: "What problem or game are variance and standard deviation optimal solutions for?". $\endgroup$ Sep 27, 2022 at 18:53

2 Answers 2

2
$\begingroup$

In econometrics, people have been using GARCH to model the conditional variances of time series with various degrees of success. The basic idea there is to impose a model for the conditional variances --- i.e. trying to "predict" them with known information. The parameters involved in the model for the conditional variance can then be estimated through various techniques using the log-likelihood as the objective.

I guess one can probably extend this line of thought and apply it to non-time-series data.

$\endgroup$
0
$\begingroup$

In short: In the general situation where the mean is not restricted to be a known constant, there exists no strictly consistent loss function for the variance. Let's unpack what that means.

(Strict) Consistency

The definition of (strict) consistency provides a suitable mathematical tool to interpret the statement 'Minimizing square loss results in predicting conditional means.' Essentially, a loss function $L: \mathbb{R} \times \mathbb{R} \to \mathbb{R}$ is consistent for the mean relative to the the class of distributions $\mathcal{F}$ if $$ \mathbb{E}_{Y \sim F} L( \mathbb{E}_{Y \sim F} (Y), Y) \le \mathbb{E}_{Y \sim F} L( x, Y) $$ for all $x \in \mathbb{R}$ and $F \in \mathcal{F}$. It is strictly consistent if equality implies $x= \mathbb{E}_{Y \sim F} (Y)$. The true expectation is thus the (unique) minimizer of the expected loss. Standard arguments show that squared error $L(x,y) = (x -y)^2$ satisfies this definition.

Naturally, we can investigate consistency for any statistic which can be computed from a distribution $F$, e.g. for the median, by simply replacing $\mathbb{E}_F (Y)$ in this definition by our statistic of interest. Also, nothing changes when we use conditional expectations instead of unconditional ones, so we can simply omit the conditioning for better understanding.

Convex level sets

The question is now whether there is a loss function $L$ such that the true variance minimizes it in expectation. Luckily, theory on strictly consistent loss functions shows that they can only exist, if the statistic of interest, let's call it $T$, satisfies the convex level sets property. This states that if two distributions $F_0$ and $F_1$ satisfy $T(F_0) = T(F_1)$, then for their convex combination $F_\lambda := \lambda F_1 + (1- \lambda) F_0$ we must have $T(F_\lambda) = T(F_0)$ for all $\lambda \in [0,1]$.

The mean ($T(F) := \mathbb{E}_{Y\sim F} Y$) and also the median satisfy this condition. The variance also satisfies this condition as long as the mean is the same for $F_0$ and $F_1$. However, in convex classes $\mathcal{F}$ where the mean is not constant, the variance will violate this property for infinitely many distribution pairs. Consequently, no strictly consistent loss function for the variance can exist relative to such general classes.

Further remarks

  • Strictly consistent loss functions are often called strictly consistent scoring functions in the literature, since the concept is closely connected to (strict) propriety of scoring rules, see also my answer to Equivalent of proper scoring rule for point forecasts. The statistics which possess strictly consistent scoring/loss functions are usually called elicitable.

  • Convex level sets are only necessary and not sufficient for existence, i.e. a statistic which has convex level sets does not need to have a strictly consistent scoring/loss function.

References

Gneiting's Making and evaluating point forecasts defines (strict) consistency and gives a proof why convex level sets are necessary for their existence (Theorem 6)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.