0
$\begingroup$

I have a predictive model $M$ that generates an empirical predictive distribution $P_M$ via a set of samples. I cannot change the predictive model.

I can evaluate the predictive performance using log score as follows

$$\sum log P_m(x) \text{ for x in validation set}$$

I've noticed that the variance of $P_m$ is too large. I can improve the log score by taking

$$\frac{P_m^{\alpha}}{\sum P_m^{\alpha}}$$

this reduces the variance and puts more mass around the center for some $\alpha > 1$.

My question is, is there any theory behind this?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.