0
$\begingroup$

In decision theory, given a family of distributions $(P_{\theta})_{\theta \in \Theta}$ on the data space $\mathcal{X}$, the risk $R_L(\theta, \delta)$ of an estimator $\delta$ for a given loss function $L(\theta, d)$ is given by $$R_L(\theta, \delta) = \int_{\mathcal{X}} L(\theta, \delta(x)) \, dP_{\theta}(x)$$ I have only seen cases where the loss function $L$ is nonnegative, making the risk function $R_L$ effectively an $L^1$-norm. Just considering the $L^1$-norm of $L$ seems a bit restrictive.

For example if we wanted to talk about the tails of $L(\theta, \delta(X))$ when $X \sim P_{\theta}$, why not use for example an Orlicz norm, $\|\cdot\|_{\psi}$? Then we could look for an estimator that minimizes $$R^{\psi}_L(\theta, \delta) = \|L(\theta, \delta(X))\|_{\psi}$$ In the case of $\psi_1$, $\psi_2$ or a similar norm, this would imply we are looking for an estimator such that $L(\theta, \delta(X))$ has (at least for a minimum assumed decay rate $\exp(-t)$, $\exp(-t^2)$ or similar) the fastest decaying tails.

I haven't been able to find anything, but I am guessing that the choice of loss function can account for this, at least to within an arbitrarily small error?

$\endgroup$
8
  • 2
    $\begingroup$ By definition, norms cannot be negative. Moreover, this formulation works for all valid loss functions, which also (by definition) cannot be negative. $\endgroup$
    – whuber
    Apr 2, 2023 at 17:15
  • $\begingroup$ @whuber What I am asking is if instead of the integral of $L$, which is the $L^1$-norm of $L$ since it is nonnegative, why not take for example $\psi_1$-norm of $L$. $\endgroup$ Apr 2, 2023 at 17:23
  • 1
    $\begingroup$ The integral is the expectation of $L.$ That's the key. $\endgroup$
    – whuber
    Apr 2, 2023 at 17:24
  • 1
    $\begingroup$ Re the edit: I can't figure out what you might mean by "the tails of $L,$" because $L$ is not a probability distribution. If you want to weight extreme values of $L$ differently then simply change $L$! $\endgroup$
    – whuber
    Apr 2, 2023 at 17:31
  • 1
    $\begingroup$ @whuber Ah so you can account for that by changing the loss. I was kind of expecting that. But that is good to know. I was thinking maybe it was because you couldn't as easily make a Bayesian connection. However, if you need just modify the loss, then that is not a problem. I will think about how to do this. Thanks! $\endgroup$ Apr 2, 2023 at 18:09

1 Answer 1

0
$\begingroup$

As mentioned in the comment by @whuber, the risk is defined as expected loss. It is not the only criterion possible, and decision theory studies many of other alternatives for making decisions, but as discussed in Why care so much about expected utility? we have have many good reasons to care about expected loss.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.