3
$\begingroup$

Suppose $x$ comes from a Bernoulli distribution $f(x, \theta) = \theta^{x}(1-\theta)^{1-x}$ where $0 \leq \theta \leq 1$. Now suppose $y$ comes from the same distribution. An estimate for the predictor distribution is $p_{\delta}(y) = (\delta(x))^{y}(1-\delta(x))^{1-y}$.

Using the Kullback-Leibler divergence, we have: $$L(\theta, \delta(x)) = D(p_{\theta} || p_{\delta}) = \theta \log \frac{\theta}{\delta(x)}+(1-\theta) \log \frac{1-\theta}{1-\delta(x)}$$

So we want to find $\min \max L(\theta, \delta(x))$. How do we do this?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.