4
$\begingroup$

FiveThirtyEight uses the following formula for their NFL Elo ratings: $$ R_i^{k+1} = R_i^k + K \cdot M(z) \cdot A(x) \cdot (S_{ij} - \sigma(x)) $$ where $z$ is the game's margin of victory, $x=R_i^k - R_j^k$, and $$ \begin{align*} M(z) &= \ln (|z|+1) \\ A(x) &= \frac{2.2}{2.2-0.001(-1)^{S_{ij}} x} = \frac{1}{1-(-1)^{S_{ij}}\frac{x}{2200}} \\ S_{ij} &= \begin{cases} 1 & i \text{ wins} \\ 0 & i \text{ loses}\end{cases} \\ \sigma(x) &= \frac{1}{1+10^{-x/400}} \end{align*} $$ Let's ignore ties.

My question is the specific justification for the $A(x)$ term: why this form, and why those numbers. (More than a layman's explanation, as found elsewhere already.)

What I (think I) understand already:

I understand that $A(x)$ is intended to maintain stationarity in the model. (This, to my understanding is not the same as correcting autocorrelation, but perhaps I misunderstand what 538 is doing.)

Intuitively, we see the function is designed so that

  • If Team $i$ is the favorite ($x>0$), a loss is upweighted ($A(x)>0$) and a win is downweighted ($A(x)<0$).
  • If Team $i$ is the underdog ($x<0$), the opposite.

However, this leaves the questions: why achieve this in this way? And why use the denominator $d=2200$?

Statistically speaking, we want $\mathbb{E}[R_i^{k+1}] = R_i^k$, i.e. we shouldn't expect Team $i$'s rating to increase --- if we did, "we should have rated them higher to begin with". I believe this issue arises because of the $M(z)$ term, because without it, we have $$ \begin{align*} \mathbb{E}[R_k^{k+1}] &= \mathbb{E}[R_i^k] + \mathbb{E}[k(S_{ij}-\sigma(x))] \\ &= R_i^k + k(\mathbb{E}[S_{ij}] - \sigma(x)) \\ &= R_i^k \end{align*} $$ which is fine, I guess.

With $M(z)$, we need $$ \mathbb{E}[M(z)\cdot A(x) \cdot (S_{ij} - \sigma(x))] = 0 $$ which, computing expectation over all possible game outcomes as encoded in $z$, given $R_i^k$ and $R_j^k$, implies $$ \int_{-\infty}^0 M(z) A(x) (-\sigma(x)) \text{Pr}(z) \ dz + \int_0^{\infty} M(z) A(x) (1-\sigma(x)) \text{Pr}(z) \ dz = 0 $$ over some distribution for $z$. Rearranging, we get $$ \frac{A(x; i \text{ win})}{A(x; i \text{ lose})} = \frac{\sigma(x)}{1-\sigma(x)} \frac{\mathbb{E}[M(z)|i \text{ lose}]}{\mathbb{E}[M(z)|i \text{ wins}]} $$ and we would want some $A(x)$ so this would hold for any $x$.

We should be able to interpolate some simple function for the expected $M(z)$'s, and then if we are satisfied with our functional form for $A(x)$, solve for the denominator $d$.

Does this analysis seem correct?

Edit. I worked out the empirical $M(z)$'s, and found they are decently approximated by the linear functions: $$ \begin{align*} \mathbb{E}[M(z)|i \text{ win}] &\approx \frac{x}{1000} + 2.2 \\ \mathbb{E}[M(z)|i \text{ lose}] &\approx -\frac{x}{1000} + 2.2 \end{align*} $$ If the $\sigma/(1-\sigma)$ term weren't there, this would give $d=2200$, but since it is there, I'm not sure what gives.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.