1
$\begingroup$

Suppose we have a random variable $Y$ with an unknown distribution $P$. We model it with a distribution $Q$. We are asked to make a point forecast under some type of loss $L$. We choose the loss minimizing $\hat y=\arg\min_y\int L(s-y) dQ(s)$ as our forecast. I am interested in the quantity

  • $\int L(s-\hat y) dP(s)$ which is the expected loss of this point forecast w.r.t. the true distribution of $Y$

and to a lesser extent in

  • $\int L(s-\hat y) dQ(s)$ which is the expected loss of this point forecast w.r.t. our model.

Question: Does any of these quantities have a name and do they play a role in any of the major decision theoretic frameworks?

I would like to learn more about their roles and intepretations; connections to economic literature are welcome.

(More generally, I am interested in decision making under uncertainty when $P$ and $Q$ do not coincide and am looking for theory/theories that account(s) for this in detail. References are welcome.)

$\endgroup$
4
  • $\begingroup$ All theory is on P. How can you build a theory on Q if it's not known? P represents what you know about Q, and you work with what you know. Also your liberal usage of "certain" and "expected" is confusing, especially in the title $\endgroup$
    – Aksakal
    Jun 25, 2019 at 15:38
  • $\begingroup$ @Aksakal, updated to address your comment. Also, we need theories that help us model reality, not theories that are easy to develop. If the theory ignores P (no connection to reality), I wonder what its use is. $\endgroup$ Jun 25, 2019 at 16:12
  • $\begingroup$ Q doesn't ignore the reality. It approximates it. It's better than having nothing at all, which is your P (unknown) $\endgroup$
    – Aksakal
    Jun 25, 2019 at 16:55
  • $\begingroup$ @Aksakal, to say that reality is nothing at all in an interesting perspective. However, if Q approximates reality, then there is a relationship between Q and P, and it is precisely this relationship that makes Q valuable in solving the problem at hand. Without a relationship to P, Q would be of no practical use. In any case, thanks for your comments and I look forward to some references. $\endgroup$ Jun 25, 2019 at 17:15

1 Answer 1

1
$\begingroup$

A good start would be a couple of papers by Tilmann Gneiting, e.g. Gneiting (2011), "Quantiles as Optimal Point Forecasts", International Journal of Forecasting, or also Gneiting (2011), "Making and Evaluating Point Forecasts", JASA. He is mainly writing about $Q$ and about minimizing the expected loss for various loss functions $L$ (although not so much about the loss itself). I have an accepted commentary on the M4 forecasting competition in the IJF where I make similar points to argue that evaluating the same point prediction with different loss functions ("error measures" in forecasting terminology) - which is very common in forecasting - makes little sense.

The pretty much accepted nomenclature among forecasters for the loss is "loss", and for the optimal point forecast is "optimal point forecast". (Sorry.) I am unfortunately not aware of any work specifically on the expected loss under the true density $P$.

I would be interested in any literature on the questions you raise, especially anything that relates the loss under the true $P$ to $Q$ or some measure of distance between the two (Kullback-Leibler looks like it might figure here).

$\endgroup$
1
  • 1
    $\begingroup$ Thanks for your edit and answer. I have read those papers a while ago and I will check them again, but as you indicate, his focus is a little off w.r.t. my interest. My problem is that I am coming from a forecasting background and trying to enter the territory of decision theory where I lack a firm footing. I was wondering, for example, whether the quantities I mention could be related to the notions of uncertainty, risk or other related terms used in some version of decision theory. $\endgroup$ Jun 25, 2019 at 17:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.