Forecasting model inputs that are both auto-correlated and are calibrated over time?

Question

How does one account for model inputs that are both a) auto-correlated and b) calibrated over time?

I'm interested in forecasting the outcomes of sporting events. Let's say that each team has a score that is assigned each week of the season that indicates how good or bad that team is. That score is updated each week based on how the team performed in that week's game. Each team's score at the beginning of the season is a "best guess" based on the previous year and their upcoming schedule.

I could easily build a historical model from all of the previous weekly scores, but something seems off about this. Scores that are assigned early in each season have greater uncertainty about them than scores assigned later in the season. Obviously, each week's score is correlated with (at least) the previous week's scores, but the scores become increasingly precise as the season progresses. Treating each score as an equal input in the forecasting model seems wrong.

What would be the best modeling approach for dealing with this? Or am I overthinking it?

Answer 1

The idea that there is greater uncertainty (randomness) in early data points suggests to me the possible need to incorporate Generalized Least Squares ( i.e. weighted estimation) when estimating an appropriate ARIMA model or Transfer Function model (if one has predictor/user specified causal/explanatory series). The whole idea is to identify a possibly useful model and then incorporate the suggestion of Tsay http://www.unc.edu/~jbhill/tsay.pdf to empirically identify the weights (diagonal adjustments to the variance-covariance matrix) after any appropriate intervention variables are incorporated in order to render the ultimate set of errors to be homogeneous. I have been reasonably successful in programming and using this approach for a number of years.

My understanding is that the ARIMA structure remedies the off-diagonal "complications" of the variance-covariance matrix thus two distinctly different weighting schemes are involved. One takes into account the time response ( i.e. ARIMA how to weight the past (moving average of the past) while the second type of weights encodes changing uncertainty/volatility/believabilty of the historical values themselves. Continuing ... the first deals with the expected value i.e. the first moment while the second deals with the second moment (variability).

Answer 2

You might start by looking at Elo rating systems used in Chess. (chess.com or letsplaychess.com have introductory articles about such systems, which began in chess but have spread to other contexts). Glickman has a lot of writing about other schemes, such as the Glicko system (Glickman's a math guy, so he likely has the technical depth you are looking for). You want to pay particular attention to how such systems handle startup conditions, because in your sports situations you never really get out of startup (except perhaps by the end of the season in baseball).

There may be other answers here in the literature of paired comparisons. What you have isn't so much a time series as a sequence of paired comparisons.

Answer 3

As a powerful, but simple framework for this sort of problem you might consider bayesian scoring.

In your particular example, let's say the "score" for each week is really just the probability, $p_i$, that team $i$ wins a game. The score could be anything really, and it would just change the distributions you would use, but I'm going with some probability between 0 and 1 because it should hopefully make for a simple example.

Then, you start the year with a prior distribution on what you think that probability for a team is. You could be very subjective about this and base it on anything you want from the previous year, but at the end of the day you would just need to boil it down to 2 parameters for the Beta Distribution, for each team. From there, each win or loss in a week constitutes an observation that allows the likelihood of this win or loss to cause your current estimate for the $p_i$ value of a team to change, becoming slowly more precise each week.

You could also use a "weighted likelihood" to favor more recent weeks over less recent ones, by simply pretending that the outcome for this week occurred $w_i$ times rather than 1 time, and for previous weeks assume a weight of less than $w_i$.

As a result of this you would have a posterior estimate for what each $p_i$ is and having a distribution for this value gives you all kinds of flexibility. If you just wanted to know which of two teams was more likely to win a game, not conditional on anything in particular, you could just look at the posterior mode of the difference of their $p_i$ values, or even more simply just compare the mean $p_i$ values for each team directly.

You might try this example of Ranking Reddit Comments from Cam Davidson-Pilon or this example on Ranking Star Ratings to see what I mean -- it's a very flexible approach to scoring and it has the added benefit of not requiring you to "make special functions" to accomplish anything. It all happens within a well-founded, probabilistic framework.

Anyways, I know this is not necessarily a "forecasting" solution but if you were trying to forecast the results of matchups then I'm not sure what the point of trying to create a score is in the first place. Unless you were going to use this score as a input to a model that predicts outcomes directly .. that would make more sense.