FiveThirtyEight uses the following formula for their NFL Elo ratings: $$ R_i^{k+1} = R_i^k + K \cdot M(z) \cdot A(x) \cdot (S_{ij} - \sigma(x)) $$ where $z$ is the game's margin of victory, $x=R_i^k - R_j^k$, and $$ \begin{align*} M(z) &= \ln (|z|+1) \\ A(x) &= \frac{2.2}{2.2-0.001(-1)^{S_{ij}} x} = \frac{1}{1-(-1)^{S_{ij}}\frac{x}{2200}} \\ S_{ij} &= \begin{cases} 1 & i \text{ wins} \\ 0 & i \text{ loses}\end{cases} \\ \sigma(x) &= \frac{1}{1+10^{-x/400}} \end{align*} $$ Let's ignore ties.

My question is the specific justification for the $A(x)$ term: why this form, and why those numbers. (More than a layman's explanation, as found elsewhere already.)

What I (think I) understand already:

I understand that $A(x)$ is intended to maintain stationarity in the model. (This, to my understanding is not the same as correcting autocorrelation, but perhaps I misunderstand what 538 is doing.)

Intuitively, we see the function is designed so that

  • If Team $i$ is the favorite ($x>0$), a loss is upweighted ($A(x)>0$) and a win is downweighted ($A(x)<0$).
  • If Team $i$ is the underdog ($x<0$), the opposite.

However, this leaves the questions: why achieve this in this way? And why use the denominator $d=2200$?

Statistically speaking, we want $\mathbb{E}[R_i^{k+1}] = R_i^k$, i.e. we shouldn't expect Team $i$'s rating to increase --- if we did, "we should have rated them higher to begin with". I believe this issue arises because of the $M(z)$ term, because without it, we have $$ \begin{align*} \mathbb{E}[R_k^{k+1}] &= \mathbb{E}[R_i^k] + \mathbb{E}[k(S_{ij}-\sigma(x))] \\ &= R_i^k + k(\mathbb{E}[S_{ij}] - \sigma(x)) \\ &= R_i^k \end{align*} $$ which is fine, I guess.

With $M(z)$, we need $$ \mathbb{E}[M(z)\cdot A(x) \cdot (S_{ij} - \sigma(x))] = 0 $$ which, computing expectation over all possible game outcomes as encoded in $z$, given $R_i^k$ and $R_j^k$, implies $$ \int_{-\infty}^0 M(z) A(x) (-\sigma(x)) \text{Pr}(z) \ dz + \int_0^{\infty} M(z) A(x) (1-\sigma(x)) \text{Pr}(z) \ dz = 0 $$ over some distribution for $z$. Rearranging, we get $$ \frac{A(x; i \text{ win})}{A(x; i \text{ lose})} = \frac{\sigma(x)}{1-\sigma(x)} \frac{\mathbb{E}[M(z)|i \text{ lose}]}{\mathbb{E}[M(z)|i \text{ wins}]} $$ and we would want some $A(x)$ so this would hold for any $x$.

We should be able to interpolate some simple function for the expected $M(z)$'s, and then if we are satisfied with our functional form for $A(x)$, solve for the denominator $d$.

Does this analysis seem correct?

Edit. I worked out the empirical $M(z)$'s, and found they are decently approximated by the linear functions: $$ \begin{align*} \mathbb{E}[M(z)|i \text{ win}] &\approx \frac{x}{1000} + 2.2 \\ \mathbb{E}[M(z)|i \text{ lose}] &\approx -\frac{x}{1000} + 2.2 \end{align*} $$ If the $\sigma/(1-\sigma)$ term weren't there, this would give $d=2200$, but since it is there, I'm not sure what gives.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.