I'm having a hard time understanding McElreath's explanation of how the KL divergence allows us to decide whether one of two models is closer to the 'real' model.
Here is what McElreath writes on p. 209 of the 2nd edition of Statistical Rethinking:
To use Dkl to compare models, it seems like we would have to know p, the target probability distribution. In all of the examples so far, I've just assumed that p is known. But when we want to find a model q that is the best approximation to p, the "truth", there is usually no way to access p directly. We wouldn't be doing statistical inference, if we already knew p.
But there's an amazing way out of this predicament. It helps that we are only interested in comparing the divergences of different candidates, say q and r. In that case, most of p just subtracts out, because there is a Elog(pi) term in the divergence of both q and r. This term has no effect on the distance of q and r from one another. So while we don't know where p is, we can estimate how far apart q and r are, and which is closer to the target. It's as if we can't tell how far any particular archer is from hitting the target, but we can tell which archer gets closer and by how much
I follow this explanation up to the point where he says "and which is closer to the target". If we don't know the target, estimating how far apart q and r tells us nothing about which one of these is closer to the target. For instance, if q is 2 and r is 10, the target can be 0 or 5 or 100 - the difference between q and r gives us no clue about where that target may be and whether q or r is closer to it.
I'm incorporating insights gleaned from the answer by @dipetkov to ask further questions.
First, if I'm understanding correctly, the formula should be:
*Dkl(p,q) - Dkl(p,r) ~ [$\Sigma$logp - $\Sigma$logq] - [$\Sigma$logp - $\Sigma$logr] = -$\Sigma$logq + $\Sigma$logr
This is different from the answer because it ignores the true probabilities ("it's an estimate of Elog(q), just without the final step of dividing by the number of observations").
The question that arises is McElreath goes on to talk about the log probability score for a particular model from hereon, even though the understanding is that the we need to compare the scores from 2 models as per the formula above.
So which one is it? The sum of the log probabilities of a given model, on its own, without comparing to another model, doesn't seem to provide us with much info, yet that's what's defined as deviance on page 210 - the sum of the log-pointwise-predictive-density multiplied by -2.
The accepted answer and further investigation made me understand that deviance is a measure associated with a single model. Then we compare differences in deviances (and perhaps the standard error of the differences in pointwise log-probability between 2 models as McElreath explains on pg. 227) to gauge the relative performance of a model.