Predictions based on data with correlations within and between multiple sets of time series

Question

I'm looking for a model to learn relationships within and between a set of partially observed time series in order to generate predictions for any timepoint in any of the set of time series.

More specifically, I have data on the proportion of available team minutes played by soccer players (range 0-1), aggregated per year of player age (e.g. 18 y.o., 19 y.o., etc.) and per (binned) Elo (i.e. skill) level of their team (e.g. 70<Elo<=80, 80<Elo<=90, etc.). The data are unavoidably incomplete - a player can only play at one Elo level (i.e. at one team) at any given moment in time - but the aim is to be able to infer what proportion of minutes a player was / is capable of playing at any team level and at any age based upon their observed minutes played thus far in their career.

Attempts so far

Factorisation machines

By representing each player's data as a MxN matrix where M=Elo level bin and N=age like so:

I can use matrix factorisation at each Elo level independently - although the results are not great; e.g. player 10 has been observed (orange points) playing at substantially higher Elo levels than player 23 (blue points), yet the predictions for player 23 at Elo level 90 (blue line) are higher than for player 10 (orange line). A smoother series of predictions at each Elo level would also be desirable, ideally reducing to zero for very low and very high ages.

I also started looking into tensor factorisation (e.g. TensorLy) to fit across all Elo levels simultaneously.

Regression

One key feature of the data remains unaccounted for with factorisation approaches: there is an evident (non-linear) correlation between age and minutes played across all players and Elo levels. Furthermore, there are correlations across Elo levels: e.g. players with a high proportion of minutes played at a medium Elo level at a young age are more likely to play a higher proportion of minutes at a higher Elo level when older. Here's a visual example:

As @leviva suggested below, we can define a regression with quadratic $age$, $Elo$ terms and an $age*Elo$ interaction term like so:

$minutes(age,Elo,player) = a_0 + a_1*age + a_2*Elo + a3*age*Elo + a4*age^2 + a5*Elo^2 + a6*age*Elo$

A nonlinear least squares (stats::nls() in R) fitted to data generates the following predictions (in blue) for these observations (in red):

I'm currently looking at modelling $player\_id$ as a random effect using nonlinear mixed effect models (e.g. nlme::nlme in R) to improve predictions by fitting independent intercepts and/or coefficients.

However, these approaches still omit information about the previous observations of minutes played by an individual player (e.g. playing many minutes at a medium team level when young should predict playing more minutes than average at a high team level when older). Therefore, I've tried including lagged terms for minutes played (i.e. proportion of available team minutes played at age t-1, t-2, etc.) and Elo (i.e. skill level of team at age t-1, t-2, etc.) like so:

$minutes_t(age,Elo,player) = a_0 + a_1*age + a_2*Elo + a_3*age*Elo + a_4*age^2 + a_5*Elo^2 + b_1*player*minutes_{t-1} + b_2*player*Elo_{t-1} + b_3*player*minutes_{t-1}*Elo_{t-1} + \ldots$

Below shows a model with lagged terms for the previous two seasons, which does help to personalise the predictions somewhat (in purple; model without lagged terms in blue).

However, this creates a jagged shift in predictions in the short-term (i.e. between the 2nd and 3rd season going forward), and worsens some predictions in the long-term (e.g. at age 35, player_id=2 is predicted to play more minutes at Elo 90 than at Elo 60). Furthermore, expanding this approach by including more lagged terms (e.g. for the past five seasons) will lead to a very large number of model coefficients which seems not ideal?

Other options

Other regression possibilities could include (i) non-linear mixed effects models (NLMMs) with the same fixed effects as the nonlinear least squares above but with player_id additionally modelled as a random effect; (ii) time series forecasting models (e.g. ARIMA) which somehow account for correlations between each time series; (iii) recurrent neural networks (e.g. LSTM) which input all previous predictions (i.e. not just the last N seasons as a lagged variable). However, all of the regression-based approaches seem to me less capable of learning player similarity information both within and between players.

Answer 1

I don't think it's a recommendation problem, but a regression one. Could you elaborate on why you think it's a recommendation problem?

In recommendation problems you usually fit a ranking of results to a query, like search engines, Netflix and Amazon recommendations, etc. the question usually asked in recommendation is: "for a query with the following features, rank the most relevant results".

The question you are asking here (as I understand it) is: "given the player and team features, what is the value of minutes played?"

This is a regression problem of the form:

$minutes = f(features_{player}, features_{team})$

The curve fitting approach is therefore the correct one IMHO. So this is how I would approach the problem:

We assume there's a surface, as you did, that a player's career follows. This is expected to be something of the following form:

1. at constant ELO, a player at age 18 should have $X$ minutes played, then it should increase as his skill improves with age, speak at some age, and then decrease as the player approaches retirement.
2. at constant age, minutes played should decrease with ELO.
3. each player has some parameter that sets their skill level apart from other players, i.e. a bias term.
4. players that have the same history should share the same curve and get the same predictions.

So one approach you can do is regression of the following form:

$minutes(age,ELO,player) = a_1*age + a_2*ELO + a3*age*ELO + a4*age^2 + a5*ELO^2 + b_{player}$

Here you represent the surface as a full second-degree polynomial and a learned bias term for every user. You can of course add more interaction terms. for example:

$minutes(age,ELO,player) = a_1*age + a_2*ELO + a3*age*ELO + a4*age^2 + a5*ELO^2 + b_{player} + b_1*player*age + b_2*player*ELO$

If you're very set on using factorization machines you can do the regression with them using the same model while replacing the interaction terms with embedding for the ELO, age, and player (see rendle).

This model though has a problem with cold start, as for every new player not in the training set, you need to fit their player parameters before being able to predict. An alternative model would be to embed the user history and then run regression on the embeddings. This could be done with an autoencoder that has an input of the user history and output that predicts all the player values (a matrix of minutes played for every ELO and age). This can also be solved with different approaches, but still as a regression problem.

Personally, I would go for the first approach as it's simpler and I believe would give good results, and then I could tinker with neural networks to solve it for fun.