1
$\begingroup$

Looking for thought partners to help me clarify a shower thought.

Let's assume I'm a forecaster, and I have in front of me several events with binary outcome–– e.g. a bent coin toss, a two-candidate election, a Premier League match between Man City and Liverpool.

What I'm looking for is a measure of "predictability". That is, very loosely, a kind of difficulty in predicting the outcome of each of these events. If I think about these in terms of a distribution of outcomes, I believe this aligns with the entropy of that distribution. If I have past data on outcomes, I could of course estimate that distribution, compute the entropy, and have a measure of uncertainty.

However, in the case where I do not have past data, and I only have knowledge of the, let's overload a term, "degrees of freedom" of these events, what can I say about my ability to predict the outcome?

That is, intuitively, a coin toss feels the least "predictable" of my examples-- its sampling process feels more "random" than the others. In the election case, there is a significant trend leading up to the event, there are many indicators/correlates to draw on, it feels much more predictable. The football match feels similarly, though the event itself, the match, is the complex arrangement and decisions of 22 people, game theory, etc. and therefore feels somewhere in between.

Of course these are hunches! I'm just giving you a feel for the concept I'm thinking about. I'm curious what tools we have which would order these examples in terms of their "predictability", or difficulty in producing an accurate forecast.

One related idea, perhaps, is parity: in the football match, parity would imply that teams, all things considered, have an equal chance of winning (e.g. due to similarities in players, strategies, information). Parity might imply a maximal difficulty / minimal predictability state-- it's random! Therefore perfectly model-able. However, a binary outcome event doesn't require parity to be minimally predictable, as in the case of the bent coin.

Thanks in advance, looking forward to your thoughts.

Edit: I think what I'm looking for might be related to Algorithmic probability though this is a topic I'm not familiar with! That said, the number (or complexity?) of causes of an event could be an important aspect of what I'm looking for.

Improve this question
$\endgroup$
5
  • $\begingroup$ Related 1 $//$ Related 2 when I deny and then provide the seed value $\endgroup$
    – Dave
    Commented Oct 10 at 13:36
  • $\begingroup$ Nice, thanks @Dave! Good discussion in 1, particularly the comparison between a "guesser" and randomization. That said, I'm still looking for input on the a priori assessment of an impending prediction. Post hoc, of course, we can review our predictions relative to alternatives and reality. But a priori, what attributes of the prediction are most relevant to assess the degree of "pure" randomness, or the bounds on how well we can predict a particular occurrence? $\endgroup$
    – spencer wilson
    Commented Oct 10 at 14:26
  • $\begingroup$ There is a tag forecastability, which you can maybe add. But its tag wiki limits its scope to time series $\endgroup$
    – kjetil b halvorsen ♦
    Commented Oct 10 at 16:11
  • $\begingroup$ Thanks! After pondering this off and on today, I think what I’m after is a subjective belief about the causes underlying outcomes, which might point to something like a Bayesian hyperprior on the causal structure of a model of the event. But again, then you get into a belief about the minimum description length of a model, which you could only adjudicate by collecting data and performing model comparison. And maybe that’s all you can do! $\endgroup$
    – spencer wilson
    Commented Oct 10 at 16:28
  • $\begingroup$ Related 3 $\endgroup$
    – Dave
    Commented Oct 11 at 1:39

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.