4
$\begingroup$

We know that in a prediction task of predicting $Y$ given $X$, $g(x) = E[Y|X=x]$ is the best predictor if the loss function is mean squared loss (albeit not the only one), $E[(y-g(x))^2]$.

For which expected loss function (or functions), conditional variance $VAR[Y|X]$ is the optimal predictor?

I acknowledge the fact that there may not be such a loss function as it may require the loss function to have a priori knowledge of the mean. And if it exists it is definetely not a proper loss function as variance is always positive but the domain of a distribution may be on the negative part of the real line. Nevertheless I cannot prove its absence.

$\endgroup$
3
  • 2
    $\begingroup$ The question is confusing. First you say you want to predict Y given X and point out correctly that for squared error loss it is the best predictor. But then you pick VAR[Y|X]. Are you still trying to predict Y? $\endgroup$ Commented Dec 18, 2016 at 1:48
  • $\begingroup$ @MichaelChernick There is nothing confusing about it. I provided the answer below, albeit with some restrictions (which were actually conjectured as part of the question). $\endgroup$ Commented Jun 26, 2017 at 7:22
  • $\begingroup$ If there is no a priori knowledge of the mean, it is possible to prove that no loss function exists, see my answer to this question $\endgroup$ Commented Nov 25, 2023 at 16:34

1 Answer 1

4
$\begingroup$

It turns out that the question has a neat answer under some conditions.

Linex Loss function is defined as follows:

$L(Y, \hat{Y}; a) = \frac{2}{a^2} (e^{a(Y-\hat{Y})}-a(Y-\hat{Y})-1)$

If we assume that $Y \sim N(\mu,\sigma^2)$ then

$\hat{Y} = argmin_\hat{y} (E[e^{a(Y-\hat{y})}]-aE[Y-\hat{y}])$

$ = argmin_\hat{y} [e^{a(\mu-\hat{y})+\frac{1}{2}a^2\sigma^2}-a(\mu-\hat{y})]$

taking the derivative with respect to $\hat{y}$ and setting it to 0 yields

$\hat{Y} = \mu + \frac{a}{2}\sigma^2$

If the mean is known to be 0, Linex Loss with parameter $a = 2$ yields the variance as the optimal predictor. Same goes for a conditional distribution, where the result is the conditional variance.

$\endgroup$
1
  • $\begingroup$ Perhaps you can address here why the result is conditional variance for a conditional distribution. I kind of get it (something about a zero-mean error term), but I’d like to see it made explicit. $\endgroup$
    – Dave
    Commented Aug 18, 2022 at 16:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.