5
$\begingroup$

If $X\sim P$, given some other distribution $Q\gg P$ what is known about $\mathbb{P}(P(X)< Q(X))$, i.e. the probability the outcome was more likely to have come from $Q$?

In particular are there any bounds on the quantity in terms of statistical distances?

$\endgroup$
4
  • $\begingroup$ What does the double greater than sign mean? $\endgroup$ Commented Apr 30, 2017 at 22:16
  • 1
    $\begingroup$ $P\ll Q$ means $P$ is absolutely continuous with respect to $Q$, which means that "all the events $Q$ says have probability 0, $P$ also says have probability 0." It's so that the ratio $P(X)/Q(X)$ makes sense. $\endgroup$ Commented Apr 30, 2017 at 22:30
  • 1
    $\begingroup$ The question in the title and the question in the body seem like pretty different questions. Also, I'm not sure what the notation $P(X)$ or $Q(X)$ mean; I'm not even sure whether $P$ and $Q$ are supposed to be PDFs, CDFs, probability measures, or something else. $\endgroup$ Commented May 3, 2017 at 16:27
  • $\begingroup$ The probability I am asking about is precisely "given $X$ came from $P$, what is the probability the outcome is more likely to have come from $Q$" which due to Neyman-Pearson is the most powerful hypothesis test for some significance level. The question only makes sense as stated if $P,Q$ are pmfs on finite alphabets. $\endgroup$ Commented May 3, 2017 at 17:26

1 Answer 1

0
$\begingroup$

For general PMFs $P,Q$ with $X\sim P$ then nothing interesting can be said about $\mathbb{P}(P(X)<Q(X))$ for all the metrics I know about:

Imagine $P(x) = \frac{1}{n}$ for $x\in[1:n]$, 0 otherwise, and $Q(x) = \frac{1-\varepsilon}{n-1}$ for $x\in [1:n-1],$ $Q(n)=\varepsilon$ and 0 otherwise. For $X\sim P$, then $\mathbb{P}(P(X)<Q(X))=1-\varepsilon-\frac{1}{n}$, even though all these distance metrics go to 0:

  • $\|P-Q\|_1\approx 2/n$,
  • $\|P-Q\|_2\approx \frac{1}{n\cdot (n-1)}$ and
  • $D(P\|Q)\leq \log(\frac{n-1}{n})-\frac{1}{n}\log(\varepsilon n)$.

With slight adjustment, the following holds in general (follows easily by expanding it into an integral of an indicator function):

\begin{align} \mathbb{P}(P(X)<c\cdot Q(X)) \leq c. \end{align}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.