Predictions based on data with correlations within and between multiple sets of time series

I'm looking for a model to learn relationships within and between a set of partially observed time series in order to generate predictions for any timepoint in any of the set of time series.

More specifically, I have data on the proportion of available team minutes played by soccer players (range 0-1), aggregated per year of player age (e.g. 18 y.o., 19 y.o., etc.) and per (binned) Elo (i.e. skill) level of their team (e.g. 70<Elo<=80, 80<Elo<=90, etc.). The data are unavoidably incomplete - a player can only play at one Elo level (i.e. at one team) at any given moment in time - but the aim is to be able to infer what proportion of minutes a player was / is capable of playing at any team level and at any age based upon their observed minutes played thus far in their career.

Attempts so far

Factorisation machines

By representing each player's data as a MxN matrix where M=Elo level bin and N=age like so:

I can use matrix factorisation at each Elo level independently - although the results are not great; e.g. player 10 has been observed (orange points) playing at substantially higher Elo levels than player 23 (blue points), yet the predictions for player 23 at Elo level 90 (blue line) are higher than for player 10 (orange line). A smoother series of predictions at each Elo level would also be desirable, ideally reducing to zero for very low and very high ages.

I also started looking into tensor factorisation (e.g. TensorLy) to fit across all Elo levels simultaneously.

Regression

One key feature of the data remains unaccounted for with factorisation approaches: there is an evident (non-linear) correlation between age and minutes played across all players and Elo levels. Furthermore, there are correlations across Elo levels: e.g. players with a high proportion of minutes played at a medium Elo level at a young age are more likely to play a higher proportion of minutes at a higher Elo level when older. Here's a visual example:

As @leviva suggested below, we can define a regression with quadratic $$age$$, $$Elo$$ terms and an $$age*Elo$$ interaction term like so:

$$minutes(age,Elo,player) = a_0 + a_1*age + a_2*Elo + a3*age*Elo + a4*age^2 + a5*Elo^2 + a6*age*Elo$$

A nonlinear least squares (stats::nls() in R) fitted to data generates the following predictions (in blue) for these observations (in red):

I'm currently looking at modelling $$player\_id$$ as a random effect using nonlinear mixed effect models (e.g. nlme::nlme in R) to improve predictions by fitting independent intercepts and/or coefficients.

However, these approaches still omit information about the previous observations of minutes played by an individual player (e.g. playing many minutes at a medium team level when young should predict playing more minutes than average at a high team level when older). Therefore, I've tried including lagged terms for minutes played (i.e. proportion of available team minutes played at age t-1, t-2, etc.) and Elo (i.e. skill level of team at age t-1, t-2, etc.) like so:

$$minutes_t(age,Elo,player) = a_0 + a_1*age + a_2*Elo + a_3*age*Elo + a_4*age^2 + a_5*Elo^2 + b_1*player*minutes_{t-1} + b_2*player*Elo_{t-1} + b_3*player*minutes_{t-1}*Elo_{t-1} + \ldots$$

Below shows a model with lagged terms for the previous two seasons, which does help to personalise the predictions somewhat (in purple; model without lagged terms in blue).

However, this creates a jagged shift in predictions in the short-term (i.e. between the 2nd and 3rd season going forward), and worsens some predictions in the long-term (e.g. at age 35, player_id=2 is predicted to play more minutes at Elo 90 than at Elo 60). Furthermore, expanding this approach by including more lagged terms (e.g. for the past five seasons) will lead to a very large number of model coefficients which seems not ideal?

Other options

Other regression possibilities could include (i) non-linear mixed effects models (NLMMs) with the same fixed effects as the nonlinear least squares above but with player_id additionally modelled as a random effect; (ii) time series forecasting models (e.g. ARIMA) which somehow account for correlations between each time series; (iii) recurrent neural networks (e.g. LSTM) which input all previous predictions (i.e. not just the last N seasons as a lagged variable). However, all of the regression-based approaches seem to me less capable of learning player similarity information both within and between players.

• Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.
– Community Bot
Commented Jul 25, 2022 at 12:32

I don't think it's a recommendation problem, but a regression one. Could you elaborate on why you think it's a recommendation problem?

In recommendation problems you usually fit a ranking of results to a query, like search engines, Netflix and Amazon recommendations, etc. the question usually asked in recommendation is: "for a query with the following features, rank the most relevant results".

The question you are asking here (as I understand it) is: "given the player and team features, what is the value of minutes played?"

This is a regression problem of the form:

$$minutes = f(features_{player}, features_{team})$$

The curve fitting approach is therefore the correct one IMHO. So this is how I would approach the problem:

We assume there's a surface, as you did, that a player's career follows. This is expected to be something of the following form:

1. at constant ELO, a player at age 18 should have $$X$$ minutes played, then it should increase as his skill improves with age, speak at some age, and then decrease as the player approaches retirement.
2. at constant age, minutes played should decrease with ELO.
3. each player has some parameter that sets their skill level apart from other players, i.e. a bias term.
4. players that have the same history should share the same curve and get the same predictions.

So one approach you can do is regression of the following form:

$$minutes(age,ELO,player) = a_1*age + a_2*ELO + a3*age*ELO + a4*age^2 + a5*ELO^2 + b_{player}$$

Here you represent the surface as a full second-degree polynomial and a learned bias term for every user. You can of course add more interaction terms. for example:

$$minutes(age,ELO,player) = a_1*age + a_2*ELO + a3*age*ELO + a4*age^2 + a5*ELO^2 + b_{player} + b_1*player*age + b_2*player*ELO$$

If you're very set on using factorization machines you can do the regression with them using the same model while replacing the interaction terms with embedding for the ELO, age, and player (see rendle).

This model though has a problem with cold start, as for every new player not in the training set, you need to fit their player parameters before being able to predict. An alternative model would be to embed the user history and then run regression on the embeddings. This could be done with an autoencoder that has an input of the user history and output that predicts all the player values (a matrix of minutes played for every ELO and age). This can also be solved with different approaches, but still as a regression problem.

Personally, I would go for the first approach as it's simpler and I believe would give good results, and then I could tinker with neural networks to solve it for fun.

• hi @leviva, thanks for the response! I've edited the question to add detail on regression approaches I've attempted so far, some of which incorporates your answer. my main problem with regression is satisfactorily accounting for previous player observations when generating predictions - this is why I considered factorisation to be an easier approach at considering all prior observations rather than having many lagged fixed effects but there's probably a better way than I've tried so far! Commented Jul 31, 2022 at 11:19
• Interesting results. Your model is now an autoregressive model (you predict $y_{t+1}$ as a function of $y_t, y_{t-1}$ ...) This differs from what I've suggested which is more similar to the factorization approach (with one dimensional factors). I suggest adding player bias and interaction: $b_{player}$ - a unique player bias $a*b_{player}*b_{ELO}$ - this is different from the term in my original answer, an interaction term between a unique player and ELO. The $b$ represents a factor for the player and ELO's $a*b_{player}*b_{age}$ - an interaction term between age and unique player Commented Jul 31, 2022 at 13:56
• You can understand this as the $b_{player}$ term is a term that takes the average curve you calculated in blue and lifts or lowers it to fit the unique player. the other interaction terms do similar effects for the ELO and age parameters. so similarity will be measured by players who have similar factors - $[b_{player}, b_{playerage}, b_{playerELO}] As for the number of parameters, it depends on the amount of data. If you have significantly more data points than parameters you can comfortably extend the model to include more factors. you should of course cross-validate the model. Commented Jul 31, 2022 at 14:05 • so just to make sure I understand, if I were defining your suggestion in the formula of a mixed effects model (e.g. lmer in R), the random effects would like so: minutes ~ ... + (1|player_id) + (1|player_id:age) + (1|player_id:Elo)? I'm not particularly familiar with mixed effects models but if I were to try and generate predictions for a new player_id with e.g. one observed datapoint, I'd have to fit the entire model again with the data of the new player? Commented Aug 2, 2022 at 12:18 • I'm not familiar with R syntax, so I'm not sure I understand it correctly but I think you formulated it correctly. However, I'm formulating it as a factorization machine. To be clear we have the following parameters:$ P_i$- a random effect for every player id$P2_i$- a factor for every player id multiplied by ELO$E2_j$- a factor for every ELO multiplied by$P2_iP3_i$- a factor for every player id to multiply by age$A3_k$- a factor for age multiplied by player id and then your personalization, or random effects, become ( for player$i$age$k$and ELO$j\$: Commented Aug 2, 2022 at 23:00