Suppose $x$ comes from a Bernoulli distribution $f(x, \theta) = \theta^{x}(1-\theta)^{1-x}$ where $0 \leq \theta \leq 1$. Now suppose $y$ comes from the same distribution. An estimate for the predictor distribution is $p_{\delta}(y) = (\delta(x))^{y}(1-\delta(x))^{1-y}$.

Using the Kullback-Leibler divergence, we have: $$L(\theta, \delta(x)) = D(p_{\theta} || p_{\delta}) = \theta \log \frac{\theta}{\delta(x)}+(1-\theta) \log \frac{1-\theta}{1-\delta(x)}$$

So we want to find $\min \max L(\theta, \delta(x))$. How do we do this?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.