I've created a pymc poisson hierarchical model to forecast sports scores.
If I use a smaller sample size of the season, say the first month, (10 games per team) versus using the entire season (100 games per team), the variance or spread between the mean of the strength of the individual teams (worst to first) is much wider than when for the larger full sample. I'm wondering how I can get an accurate forecast using a smaller sample of the season as the full season will not be available during forecasting.
And although, yes, this is a classic case of the law of large numbers shrinking my posterior as expected (see link):
Where in the above article it mentions to the effect, "yes we now know it happens, the amount can be predicted using this general formula, so at least its correctable." But how could this be incorporated into a bayesian model? Or more generally, how can I get an accurate realistic forecast from Bayes models generated from smaller sample sizes? It seems that this would plague all such bayes type models' posterior sampling.
If I simulate using the smaller posterior, my predictions show a larger variance than is true in reality. In fact I'm not sure the full season doesn't need regressing as well.
Must I regress each forecast based on the size of the sample generated? That doesn't make much sense from a Bayes perspective. I'm more of a programmer than a stats guy. I have a few solutions I can use as a workaround, keeping the variance stable for example during the pymc sim, but I'm trying to wrap my head around how to deal with this from a true statistical perspective using Bayes modeling. Any help appreciated.