All Questions

Filter by
Sorted by
Tagged with
2
votes
0answers
223 views

If the AIC is an estimate of Kullback-Leibler Divergence, then why can AIC be negative when KL divergence is always positive?

I have read many times that the AIC serves as an estimate of the KL divergence, and I know that AIC can be a negative value (and have seen that myself). Yet, the KL divergence must always be positive. ...
1
vote
0answers
306 views

Comparing ELBO of a VAE for different samples

I am lacking of an interpretation of the evidence lower bound (ELBO), when comparing two different samples $x_1, x_2 \sim X$. Writing the marginal log-likelihood as the sum of lower variational bound ...
1
vote
0answers
87 views

Kullback-Leibler divergence with sample data likelihood [duplicate]

I'm trying to get my head around the KL divergence in the context of the sample likelihood under two competing hypotheses, one optimal $H_0$ and one suboptimal $H_1$. Roughly speaking, I want to see ...
3
votes
0answers
80 views

Properties of Average Multinomial Likelihood

I am trying to understand the Kullback-Leibler Information: I read in http://arxiv.org/pdf/1404.2000v1.pdf the following: Ideally, we want the probability to be invariant to the number of ...
3
votes
0answers
183 views

Formal statistical test for comparing likelihood distributions obtained via MCMC

I am trying to formally compare the distribution of the likelihood values generated using two different models with marginal posterior values of the parameters obtained using MCMC in order to assess ...
3
votes
1answer
211 views

Property of KL-divergence

Let $p_1$ and $p_2$ be two distinct probability distributions. Define $$ L(q)=D(q||p_1)-D(q||p_2) $$ where $D$ is the usual Kullback-Leibler divergence. Assume the support of $p_2$ is included in ...