Test predictions of football scores against a null model of "no predictive skill" So, I am trying to prove that I can predict football scores better than randomly assigning win,draw or loss. I have predicted 140 scores, I have succeeded at 67. My null hypothesis is that P(Success)=(1/3), alternative hypothesis that P(Success)>(1/3),I am happy for a significance level of 1%, but I can't remember how to do it.
 A: If you want to know whether you did better than chance, compare yourself against an uninformed predictor.  Two such predictors are of interest.  In any game, choose one of the competitors by flipping a fair coin and then


*

*Guess randomly that the chosen team will win, draw, or tie, with chances of $1/3$ in each case.

*Always guess that the randomly chosen team will win.
Let's consider what Predictor 2 could accomplish.
In the 2013-14 English premier league very close to $20\%$ of games were drawn.  Thus, by always guessing that a randomly chosen competitor will win, Predictor 2 would expect to succeed close to $(100\% - 20\%)/2 = 40\%$ of the time.  Their performance over $140$ games would vary for two reasons: due to the outcomes of the coin flips (which are random and independent) and due to the choices of the $140$ games.  Since we don't know how you chose those $140$ games, let's assume they were randomly selected from all the games (just to serve as a frame of reference for this calculation).  Under this assumption, Predictor 2 would expect to be successful in $40\%$ of the games you chose.  The number of successes would have a Binomial distribution with probability $0.40$ over those $140$ games.  The chance of being successful $67$ or more times--thereby equaling or exceeding your performance--can be computed as approximately $2.4\%$.
This result is sensitive to the assumption that the $140$ games were selected randomly. Likely they were not.  What if, for instance, they usually consisted of games between strong and weak teams? Predictor 2, who remains ignorant of which teams are strong and which are weak, would do better than before simply because ties would be rarer.  For instance, if the proportion of ties in this set of games was only $10\%$, then Predictor 2 would have a $45\%$ chance of winning.  Now the Binomial distribution calculation, using $0.45$ instead of $0.40$, gives Predictor 2 almost a $28\%$ of doing as well or better than you did!
It looks like identifying a performance that is significantly different from Predictor 2 might be difficult with a record of only $140$ games.  However, as the number of contests you predict grows, any persistent difference between your performance and that of an ignorant coin-flipper will gradually become more detectable.
The upshot is that statistical calculations can be useful in exposing how your choice of games to predict can influence your assessment of how well you have done.  If you wish to go further in your assessment, then a deeper analysis of those particular 140 games would be warranted, using the actual rate at which draws were observed.
A: I agree with whuber --- would also state that the odds of predicting a win/ loss/ draw changes again if one is even roughly aware of the relative strength of the teams. Picking a win or loss is different than picking above or below the "Vegas Spread" --- the reason for such an existence of said spread (or horse betting odds) -- is due to a free market to an extent (in horse race, entirely the free market --- odds literally change as wagers and betting pools for X horse change --- while the horses themselves and "true model odds" of one winning do not change).
So what -- what does the "Free market" determining betting odds/ the predicted score spread have to do with calling winners?
Well, it turns out, as in predicting the stock market, the free market is extremely adept at forecasting, and nearly impossible to beat systematically using currently known forecasting methods. In other words, in the NFL, if you always picked the Vegas favorite to "win" --- which anyone can do --- you would fare much better than 50% success rate on average. Of course, such bets are not as common, and pay out less than double your money.
