I have a predictive model $M$ that generates an empirical predictive distribution $P_M$ via a set of samples. I cannot change the predictive model.

I can evaluate the predictive performance using log score as follows

$$\sum log P_m(x) \text{ for x in validation set}$$

I've noticed that the variance of $P_m$ is too large. I can improve the log score by taking

$$\frac{P_m^{\alpha}}{\sum P_m^{\alpha}}$$

this reduces the variance and puts more mass around the center for some $\alpha > 1$.

My question is, is there any theory behind this?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.