# To compare two KL-divergence scores, does the prior model have to be the same for both?

The KL-divergence compares a theoretical model $$p$$'s distribution with the empirical model $$q$$'s distribution, giving a score of $$0$$ if they, or their information contents, are identical.

Say we have two KL-divergences for two distinct models (that have the same units and context), therefore two pairs (4) distributions

1. If the first score $$KLD_1=0.3$$ tells us how far off model 1 ($$p_1$$) is in reality from theoretical model 1 ($$q_1$$),
2. and the second score $$KLD_2=0.4$$ quantifies how far off model 2 is in reality ($$p_2$$) from theoretical model 2 ($$q_2$$),

can I compare the values of $$KLD_1$$ and $$KLD_2$$ directly and conclude "model 1 outperformed model 2 because its score was lower, implying less divergence from its own theoretical prior" even though the scores were respectively measured based on two different models/priors? Or are two KLD scores only comparable if based on the same underlying theoretical model, i.e. $$(p_1 || \mathbf{q_1})$$ vs $$(p_2 || \mathbf{q_1})$$?

The question more abstractly,

1. KLD(red apples || green apples) vs KLD(red grapes || green grapes), or
2. KLD(red apples || mangos) vs KLD(red grapes || mangos)?