1
$\begingroup$

I am building a simple random forest to predict soccer results in sckit. I simply train the model to predict each teams score based on various features. However I am trying to think how I can weight the data so that more recent fixtures will be considered more than historic fixtures.

Any ideas?

$\endgroup$
2
$\begingroup$

The random forest should pick this up automatically from the data itself. If data closer to the present has a stronger effect on your y predictor than earlier lags, then the coefficients will be larger for more recent lags and smaller for earlier lags. The regression coefficients, i.e., "weights", should be wholly determined by the data. If you do want to try a weighted time series, you can use Holt Winters Triple Exponential smoothing on your score variable and then see if a regression model with additional lagged features beats the time series model.

$\endgroup$
  • $\begingroup$ How would the RF know about lags? $\endgroup$ – S. Kolassa - Reinstate Monica Jan 28 '16 at 20:43
  • $\begingroup$ How would it not? Random forest is just like any other regression model - it generates an answer using however many lags of variables that you feed into it. $\endgroup$ – Hidden Markov Model Jan 28 '16 at 20:44
  • $\begingroup$ I don't see how Holt-Winters - or indeed any classical time series algorithm - would help with predictions based on features. $\endgroup$ – S. Kolassa - Reinstate Monica Jan 28 '16 at 20:45
  • $\begingroup$ I guess you never heard of ARIMAX models then .... $\endgroup$ – Hidden Markov Model Jan 28 '16 at 20:45
  • $\begingroup$ Uh, no. You would at least need to transform your data and feed the lags in in some way. RF's don't automatically transform input data, at least not the implementations I'm familiar with. Maybe you could edit your answer to give an example? $\endgroup$ – S. Kolassa - Reinstate Monica Jan 28 '16 at 20:46
2
$\begingroup$

I agree with user Hidden Markov Model, when the underlying phenomena, which is generating the time series, is constant. On the other hand, if the dynamics of wins and losses transform as new football tactics appear, then very old time series are less representative of tomorrow than recent observations.

As S. Kolassa points out, time series cannot be plugged directly into RF or any other supervised regression method.

Typically for RF, a time series is treated with a rolling window generating learning examples how some past events ($X_{t-1}$ to $X_{t-k}$) coincided with some future outcome $X_t$. The model is free to up weigh any $k$ recent periods in the windows. But the regression model does not up weigh those learning examples/windows where $t$ is closest to present day by default. One can help RF up weigh recent examples by stratification. E.g. for each tree is bootstrapped(with replacement) 200 learning examples within last 200 periods of $t$ + 200 learning examples from the last 1000 periods of $t$.

Thus, when the underlying system could be transient it would make good sense to down sample distant-in-past learning examples/windows.

If your system of interest is both transient and noisy you're in trouble.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.