What value (cutoff) of KL divergence signifies that the distributions are different

Question

As per my understanding KL divergence can be used to check the divergence between two probability distributions, but is there a cutoff value of KLD, above which we say the distributions are different and below which we say they are same?

Edit: I understand 2 distributions are same if any only if kld is 0, but is there a statistical check like hypothesis checking and p values. for example the p value need not be 0 but even if p <0.05 we treat the means/proportions are different in hypothesis testing

Here is the usecase, I have in mind for using KLD to compare distributions. say I am doing an AB test, I randomized users into Test and control, wanted to check if the proportion/distribution of different covariates (age, usage type) in test and control are same and for this, I wanted to use KLD to check for the distribution divergence and looking to see if there is a cutoff above which we can say the distributions are different.

Answer 1

Two things you might think of, but that don't work

1. $P$ is a distribution, $\{X_1,\dots,X_n\}$ a set of $n$ iid observations giving empirical cdf $\mathbb{P}_n$. What is the distribution of $KL(\mathbb{P}_n,P)$ (or the reverse) when the data are sampled from $P$, and is the observed $\mathbb{P}_n$ consistent with that

2. $\{X_1,\dots,X_n\}$ and $\{Y_1,\dots,Y_m\}$ are each an iid sample from some distribution, with empirical CDFs $\mathbb{P}_X$ and $\mathbb{P}_Y$ respectively. Is $KL(\mathbb{P}_X, \mathbb{P}_Y)$ consistent with them being sampled from the same distribution?

The reason these don't work is that (at least for continuous underlying distributions) the KL divergence will typically be infinite because the distributions aren't mutually absolutely continuous. More precisely, any continuous and any discrete distribution have infinite KL divergence, and any two empirical distributions have infinite KL divergence if each one has a value that doesn't occur in the other.

In situations with discrete data and large enough sample size, where you can compare two empirical distributions or a theoretical distribution and an empirical distribution, this will reduce to something close to the multinomial likelihood ratio test.