I am currently trying to build a better forecasting model for my fantasy baseball roster. I currently am using commonly accepted projected season statistics (ZiPS from Fangraphs) to determine the average fantasy points a player can be expected to contribute per game. This is problematic, however, because it does not take into account variance in player performance (among other things).

Since baseball involves both luck and skill, I don't think it is a useful exercise to try and predict any particular statistic of a game (i.e. how many hits Prince Fielder will have). Instead, I would like to project average point contributions but take variance into account while doing so.

The first thing that comes to mind is the effect of the opposing pitcher. My hypothesis is that the quality of the pitcher effects opposing player performance. Given two players to choose from which are relatively equal in projected fantasy point averages, how can I quantify the effect of the opposing pitcher and how can I test the hypothesis?

Also, how can I consider variance in a reasonable way even though it is uncertain? How would I actually know if my projections are under performing or over performing? (This seems to be similar to a financial portfolio optimization problem)


Accounting for variance

There's a lot to think about in optimising a lineup for fantasy sports. You're right that expectation and variance are huge parts of it. Naively it would seem that expected points earned is all that matters. However, certain contests will reward only the first place player out of thousands -- meaning you want very high variance teams so that your tail is fat enough to make a win possible*. Other contests reward all those in the top half equally, in which case you care more about expectation and less about having a tail -- you just want to get the maximum amount of your expected distribution above the halfway cutoff.

For some intuition on how your lineup ties to variance, there are a couple of mechanisms. Certain players have a higher variance than others. Players on teams who never lose, for instance, tend to have pretty low variance. The main effect comes from picking players on the same team or picking players on opposing teams. Players on the same team have very high covariance -- they tend to all win or lose points together. Players facing each other have negative covariance -- either one does well or the other, but rarely both or neither. (This is magnified for certain positions, like striker vs opposing goalie in hockey or, I imagine, pitches vs opposing batters in baseball). Lineups featuring players with a high covariance to one another are lineups with a high variance.

You're right that calculating the variance of a team has analogous to portfolio optimisation. The problem is exactly the same: assets are now players, expected returns are now expected points and variance is variance.

Calculating the variance for a lineup given a set of players with known covariance is easy. If we are considering $N$ players, define $p$ as a binary vector of length $N$ with value 1 if a player is present on a team and 0 otherwise. $\Sigma$ is the $N*N$ covariance matrix of all players, then the variance of the lineup $\sigma$ is given by:

$ \sigma = p^T \Sigma p $

Where $^T$ is the transposition.

Covariance should not be calculated at an individual player level since there will never be enough data. Pitcher A's covariance with Batter B across an entire season is not useful, since they won't play each other in every game. Instead calculate it on a position level: the covariance of home team pitchers versus visiting shortstops, home team shortstops versus visiting pitchers, home team shortstops versus home team pitchers, etc. (Disclaimer: opinion, you could also calculate it purely as home vs away, or purely as home vs away and pitchers vs non-pitchers.)

Effects of opposing players

To the expectation-predicting point of your post, you should absolutely use information from the pitcher to predict the performance of a batter. Fantasy aficionados call this "defense against position" or "fantasy points against". Essentially the idea is to use the historic point earnings of batters who have faced this pitcher to predict future performance for all batters who face them. It's a hugely powerful predictor, and you can validate as you would any other model change.

* There's a really cool paper on a bunch of guys from MIT optimising drafts for very top-heavy fantasy sport contests where they plan to make multiple entries and want to maximize the expectation of their entire multiple entry strategy. They care about having high variance within the lineups they enter, but also low correlation between their entries. They treat it as a max coverage problem and solve it with integer programming. It's a good read.

  • $\begingroup$ <plug>I wrote a post on things I learned trying to model fantasy performance which goes into more detail. <\plug> $\endgroup$ – Dex Groves Sep 26 '16 at 22:33
  • $\begingroup$ Wow I didn't expect any other answers after three years... Thanks! $\endgroup$ – Aketay Sep 26 '16 at 22:46
  • $\begingroup$ You got bumped by Community a few days ago. $\endgroup$ – Dex Groves Sep 26 '16 at 22:47

Running with your example of picking between two players given the knowledge of an opposing pitcher, I think you could build a reasonable model using historical data to simulate outcomes. For example, suppose you are deciding to whether to start Player A or Player B. Player A is a 31 year old RH batter facing a RH starting pitcher that is the ace in his team’s rotation. You could create a sample of observations from the fantasy points Player A generated when he faced a strong RH starting pitcher over the last 200 games. For example, Player A may have faced 30 top ranked RH pitchers over the last 200 games so you would have 30 different fantasy point observations in your sample. 200 is totally arbitrary, you could use more or less history depending on how much of Player A’s history you feel is correlated to his recent performance. The more history you use the more games that will match the specifications of your sample, but some of the games might not be as applicable to Player A’s current skill. You could also use other players that are comparable to Player A’s skills, batting style, and age to increase the number of observations in your sample. Nate Silver talks about using comparable players in his book The Signal and the Noise. Furthermore, you could refine the games you add to the sample based on the ballpark the game is being played at (hitters park v. pitchers park), the weather (wind blowing in or out), the point in the season (early, mid, late), the type and frequency of pitches the pitcher throws, the quality of the team’s bullpen, ect…You could make the sample as specific and you like, although you might end up with only a few observations which would degrade the usefulness of simulating outcomes.

Once you have a sample of games to draw from you can create a distribution of fantasy points that Player A might generate by bootstrapping the observations. If you repeat this same process for Player B and the pitcher he is facing you will have two distributions of fantasy points that each player is likely to generate in the upcoming game. You can compare the two distributions using statistical methods for comparing two means (t-test for normal distributions, Mann-Whitney U test for non-normal). If you are comparing multiple players you would need to use methods for comparing more than two means. As far as tracking the performance of your projections, the p-values derived from the comparison of means equate to the likelihood you would expect the player with lower projected fantasy points to outperform the player with higher projected points. If you make 10 projections, each that had a 10% p-value when you did the mean comparison, then you should expect to get about 9 of these projections right. If you are getting a lot more projections wrong then the p-values from your mean comparisons would suggest something might be wrong.

This is a very brute force method for comparing two players and it relies heavily on historical data, but you could always build in some adjustment factors to tweak the ending distributions based on domain specific knowledge. For example, maybe you know that the player who bats right before Player A is on the DL and his replacement is not as good at getting on base. This will likely hamper Player A’s RBI production so maybe you want to shift the sample distribution of Player A’s expected fantasy points to the left to account for the likelihood that Player A's fantasy point generation will be less due to a lower RBI potential. You could even estimate this adjustment factor from data. All in all, there are a lot of things you can do by bootstrapping historical outcomes that you think are representative of the games you are interested in and comparing the resulting distributions.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.