I'm working on building a predictive model for the number of singles a hitter in baseball generates over the course of a single game. Since the number of singles a hitter scores per game is count data I figured that the most ideal model would be a poisson regression. While the poisson model works fairly well, when I cross validate my model the RMSE is always lower with a gaussian assumption. This makes me feel like there is some adjustment I need to make to my data or my assumptions but I cant for the life of me figure out what that might be.
From what I can tell my data fits a poisson assumption very very well. When I plot the empirical density of singles over a poisson distribution with lambda equal to the sample mean I get an almost perfect overlap so I don't suspect zero inflation. Adjusting the link assumption from log to sqrt improves the accuracy somewhat but the gaussian model still outperforms the poisson. I'm working with around 4400 data points and only using around 5-10 predictors so I don't suspect sample size is an issue. Another potential issue I suspected was overdispersion but using a quasi-poisson and negative binomial model only increased or maintained the RMSE. Finally, when I plot the OLS residuals they clearly do not satisfy the normal assumption of a gaussian distribution. (Screenshots included below)
Why could this possibly be the case? Is the lower RMSE from a gaussian assumption suggestive of a problem with my data or model? Just for reference the difference in RMSE is usually around the .02 range. My only goal is to minimize the RMSE of my model, if OLS is better at reducing RMSE would it be good practice to just ignore the clearly wrong assumptions it makes?