3
$\begingroup$

From statistical decision theory we know that if we want to minimize EPE (Expected prediction error) it is sufficient to minimize the conditional expectation of the loss function.

$f(x) = argmin_{c} E_{Y \vert X}(L(Y, c) \vert X = x)$

For square loss $L(Y,c) = (Y-c)^2$ the solution is the conditional expectation $c=E(Y \vert X=x)$.

What is an example of a loss function that is not minimized by the conditional expectation? What is it minimized by? It would be great if the example were a loss function that is actually used to some extent and not totally contrived, but everything is welcome.

I think the property of a loss function being minimized by the conditional expectation is known as being p-admissible.

$\endgroup$
0

1 Answer 1

4
$\begingroup$

If the loss function was for example $L(Y,c)=|Y-c|$

then the conditional median would minimise it

i.e. $c$ such that $\mathbb P(Y \le c\mid X=x) \ge \frac12$ and $\mathbb P(Y \ge c\mid X=x) \ge \frac12$

$\endgroup$
1
  • 2
    $\begingroup$ Extending this, we could use any of the (other) quantile regression loss functions so that the conditional mean does not optimize the loss, even when the error term is symmetric. $\endgroup$
    – Dave
    Commented Jun 6, 2021 at 12:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.