7
$\begingroup$

I want to predict PGA golfer performance. I'm wondering if I am correctly giving more weight to more recent results by using the weights= option in the lmer() function.

I have data from 2012-2014 laid out thusly:

library("lme4")
library("dplyr")

head(rdDat)
Source: local data frame [6 x 5]
Groups: plrF, trnF

  plrF trnF rdF wt rdScr rdPar
1    5  996  R1  1    71    71
2    5  996  R2  1    69    71
3    5  996  R3  1    70    71
4    5  996  R4  1    69    71
5    5  998  R1  3    72    72
6    5  999  R1  4    73    70
  • plrF - Player ID
  • trnF - trounament ID
  • rdF - round of tournament (each tournament has 4 rounds)
  • wt - Weight. Basically number of weeks since Jan 1, 2012.
  • rdScr - observed score for a golfer
  • rdPar - par for that round.

I want to use lmer() to model player scores based on a random player effect, and fixed par effect. Let's split the data into a training set and testing set.

oRdDat <- rdDat %>% filter(wt <= 120)
newdat <- rdDat %>% filter(wt > 120)

Fit a model on the observed data:

lmr1 <- lmer(rdScr ~ rdPar + (1 | plrF), data= oRdDat)

Use the results to predict the new data, and calculate the absolute error of our prediction:

pred0 <- cbind(newdat, prScr = predict(lmr1, newdat, allow.new.levels = TRUE)) %>% 
  mutate(diff = abs(prScr - rdScr))

and use that diff variable to check the Mean Absolute Error of our projection:

summary(pred0$diff)[4]
 Mean 
2.481  

However, I think it is very reasonable to assume more recent results (eg late 2014) should have more of an impact on our projection than results from early 2012. So I fit this:

wlmr1 <- lmer(rdScr ~ rdPar + (1 | plrF), weights = wt, data= oRdDat)

Predict as before and check out the MAE:

summary(pred1$diff)[4]
 Mean 
2.474 

Incremental improvement! :-D

Let's leave aside the question of what the optimal weighting scheme would be and whether the small improvement seen here is actually 'worth it'. My question is: Is that weights=wt option doing what I want it to do? Eg, provide more weight to more recent results in terms of projecting future scores?

Thanks!

$\endgroup$
6
$\begingroup$

The log-likelihood is defined as: $$ \log(L(\boldsymbol{\theta})) = \sum_{i = 1}^{n} w_i \log(P(y_i | \boldsymbol{x}_i, \boldsymbol{\theta})) $$ where $\boldsymbol{\theta}$ are the model parameters, $w_i$ is the weight for observation $i$, $y_i$ is the response for observation $i$, and $\boldsymbol{x}_i$ is the vector of covariates for observation $i$. So, yes, I think the weights option is doing exactly what you want - the more recent observations have a greater contribution to the log-likelihood.

I know you specifically didn't ask for any comments on this in your question, but Dixon and Coles considered using such weights in order to increase the predictive performance of their soccer model - so might be worth a look at using a similar weighting function (if you are not already familiar with this).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.