# All Questions

6 questions
Filter by
Sorted by
Tagged with
223 views

### If the AIC is an estimate of Kullback-Leibler Divergence, then why can AIC be negative when KL divergence is always positive?

I have read many times that the AIC serves as an estimate of the KL divergence, and I know that AIC can be a negative value (and have seen that myself). Yet, the KL divergence must always be positive. ...
306 views

### Comparing ELBO of a VAE for different samples

I am lacking of an interpretation of the evidence lower bound (ELBO), when comparing two different samples $x_1, x_2 \sim X$. Writing the marginal log-likelihood as the sum of lower variational bound ...
87 views

### Kullback-Leibler divergence with sample data likelihood [duplicate]

I'm trying to get my head around the KL divergence in the context of the sample likelihood under two competing hypotheses, one optimal $H_0$ and one suboptimal $H_1$. Roughly speaking, I want to see ...
80 views

### Properties of Average Multinomial Likelihood

I am trying to understand the Kullback-Leibler Information: I read in http://arxiv.org/pdf/1404.2000v1.pdf the following: Ideally, we want the probability to be invariant to the number of ...
Let $p_1$ and $p_2$ be two distinct probability distributions. Define $$L(q)=D(q||p_1)-D(q||p_2)$$ where $D$ is the usual Kullback-Leibler divergence. Assume the support of $p_2$ is included in ...