Definition of "unpredictable"

Question

How do we rigorously define the term "unpredictable" in cases of point and density prediction?

The term "unpredictable" is employed in various contexts, e.g.

• "the outcome of a toss of a fair coin is unpredictable",
• "increments of a random walk are unpredictable" or
• "in an informationally efficient market, price changes are unpredictable".

These statements are not literally true, as one can always provide a prediction, however inaccurate. I can make a point prediction (the coin will turn up heads; the increment will be 0.18879; the price change will be £0.12) or a density prediction ($P(\text{heads})=0.51$, the increment is N(0,0.4), the price change is ...).

Intuitively, "unpredictable" means something like "cannot be predicted more accurately than by some simple/naive/natural benchmark" such as "the best density prediction for the coin toss is $P(\text{heads})=0.5$". But then we need to define what we mean by "best" when evaluating density predictions.

So, how do we rigorously define the term "unpredictable"?

_{Keywords: unpredictable, unforecastable, predictability, forecastability.}

Answer 1

I see several levels to unpredictability.

1. Positive variance: If the outcome is uncertain, there is a degree of unpredictability. If the outcome is certain, then there is no unpredictability.
2. Positive variance, but zero conditional variance: Maybe an outcome is uncertain, but it becomes certain once you observe some other information (e.g., regression features).
3. Can’t do better than predicting based on a uniform distribution on the space of outcomes: If the distribution gives equal likelihood to every outcome, then you truly have no idea about which will happen. (I’m not quite sure how to think about this when the space of outcomes is unbounded.)
4. Can’t do better than always making the same naïve prediction, such as always predicting the majority category: Then there is a limit to how good the predictions can be, and it takes minimal skill to reach that limit (basically no skills required to buy an S&P 500 index fund).

I think the last one seems consistent with efficient market hypotheses. We might think it is impossible to do better, consistently, than buying and holding the entire S&P 500, but we do not claim that buying and holding the S&P 500 is foolish or has zero expected gain. Indeed, doing so seems to be a wealth pump!

Bringing rigor to the last two requires a few definitions to be made.

DO BETTER is defined by some kind of utility function that will depend on what is valued. In the context of finance, perhaps this is the returns on an investment.

NAÏVE PREDICTION requires a careful definition of a benchmark or baseline model, as is mentioned in a nice comment by Stephan Kolassa.

My 2 cents: I'm glad I don't have to deal with this question... I don't use the term "unpredictable", and I don't know whether it's a good term to have in our vocabulary. I would prefer something like "can't be consistently predicted better than using insert simple benchmark prediction". This immediately gives the benchmark to compare to and draws attention to the fact that we can indeed always give some kind of prediction.

Answer 2

I think you're making this more complicated that it needs to be. "the outcome of a coin toss is unpredictable" - mean exactly what it says, that the outcome is random and is not determined fully by the state of the world before the coin toss. If you're interested in the probability, i.e. population parameter, then it is predictable: it's whatever you measure it because it doesn't change.

When we say the stock price is unpredictable we mean just that: we can't beat the random toss. It doesn't mean we can't calculate the distribution parameters with some precision.