Forecasting a distribution for count data I'm working on a project to forecast the distribution of a baseball player's "At Bats per game" (a baseball statistic w/ integer domain) using a player's position in his team's batting order as a predictor (1st up to bat, 2nd up to bat etc). 
I went with my initial thought, which was to train a logistic regression on historical data w/ a poisson assumption, plug in the batting order for the game I want to predict and just nab the predicted lambda from the model without actually attempting to predict a value. 
I then decided to plot the density of my "fitted" poisson model over the empirical density from my data. Every time I do this the poisson distribution has much more variance than the empirical density. I have several thousand data points so I suspect my empirical density should be pretty good.
I've attached a picture of the empirical density(proportion of all at bats for players with a batting order = 2) plotted against a poisson with lambda predicted for batting order = 2 from my glm. Is there another technique I can use to better predict a distribution of count data? I'm assuming a negative binomial glm will give roughly the same results. I know it's somewhat taboo but it seems like I could get a much better fit for the distribution using a normal distribution from an ols. Thanks for any advice! 
 A: This is probably the first time I have seen underdispersion in practice. Usually the problem is that observations are overdispersed with respect to the model.
Now, if data are underdispersed compared to a Poisson distribution (with equal mean and variance), I'd immediately think of a binomial distribution, where the variance $np(1-p)$ is always smaller than the mean $np$.
You could do a "kind of ad hoc binomial regression": parameterize $p=\frac{1}{1+e^{-\beta x}}$ with your regressors $x$ as in the logistic case, to constrain $0<p<1$. Set up the log-likelihood and estimate $\beta$. Do this separately for $n=1, 2, \dots$, yielding estimates $\hat{\beta}^{(n)}$. Pick the $(n, \hat{\beta}^{(n)})$ combination that yields the highest overall log-likelihood.
If your predictors are truly discrete and can only take on a finite number of values, you may instead want to group your data and estimate separate $(\hat{n}_x, \hat{\beta}_x)$ for each observed value of $x$ (then simply $\hat{p}_x = \frac{1}{1+e^{-\hat{\beta}_x}}$).
This is extremely ad hoc. Typically, in binomial regression, your $n$ is set a priori, and you are only interested in estimating your $p$ - here, you'd have to estimate both, and the problem is that there is no good interpretation of $\hat{n}$ (nor of $\hat{p}$). The only two good points this approach would have would be that (a) it's inherently discrete, in contrast to the normal approach that you mention, and (b) it should be able to capture the underdispersion. It still won't capture the negative correlation between innings (which you could potentially take into account by taking previous at-bats into your regressors), though. But it might improve on your model so far.
