9
$\begingroup$

The answer to What is the probability P(X > Y) given X ~ Be(a1, b1), and Y ~ Be(a2, b2), and X and Y are independent? provides an analytical solution for this, but is there a less computationally intensive expression if one is willing to lose some precision?

Assume integer a and b, if that helps.

Improve this question
$\endgroup$
2
  • $\begingroup$ Numerical integration would be my first thought. Simulation would be my second. The numerical integration would probably involve trying to use quadrature based on Jacobi polynomials. Neither of those are analytical though. $\endgroup$
    – Glen_b
    Commented Mar 26, 2013 at 5:49
  • $\begingroup$ Both use quite a lot of iterations. I considered using precomputed tables, and interpolating, but this would use too much memory with inaccurate results. $\endgroup$
    – Andris Birkmanis
    Commented Mar 26, 2013 at 15:18

2 Answers 2

Reset to default
8
$\begingroup$

Use this close-form approximation approximation. The author claim it is 2 orders of magnitude faster to evaluate.

To abide by CV rules I will be citing key infos from the intro of this very clear paper:

The key idea is that:

The key idea in that paper is that

$$P(X_B>Y_B)\approx P(X_N>Y_N)$$

where $X_B$ and $Y_B$ are independent beta random variables and $X_N$ and $Y_N$ are their normal approximations formed by moment matching. The author shows that these approximations are rather accurate, even for small values of $(a_1,b_1)$ and $(a_2,b_2)$. For example: when these parameters take integer values between 1 and 10 inclusive, the average absolute error is 0.006676.

Edit:updated link.

Improve this answer
$\endgroup$
3
  • $\begingroup$ This is exactly what I needed! Fast and sufficiently accurate! $\endgroup$
    – Andris Birkmanis
    Commented Mar 26, 2013 at 14:52
  • 3
    $\begingroup$ Unfortunately, that paper doesn't show anything: it only makes a claim. The average absolute error is encouraging but not terribly useful, because the cases with $a_i\approx b_i$ are going to be well approximated by the Normal distribution. The tough cases are the skewed ones with very different values of $(a_i,b_i)$ or with very small values of at least one parameter. $\endgroup$
    – whuber ♦
    Commented Aug 9, 2020 at 13:46
  • 1
    $\begingroup$ @whuber thanks for your comment. I was merely updating the dead link. I think the author studies in fact the range of values of $a_i$ and $b_i$ for which the normal approximation works. The author indeed points out the need for an alternative approach when $a_i$ is very different from $b_i$ but does not offer any. $\endgroup$
    – user603
    Commented Aug 9, 2020 at 19:33
1
$\begingroup$

I'm uncertain if this is answering the question you are asking. But you may want to check out Asymptotics of Evan Miller's Bayesian A/B formula. What they effectively are solving for is P(Conversion Rate of A > Conversion Rate of B) given two Beta distributions.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Welcome to the site! You can improve this answer by expanding it, perhaps by giving a summary of the information at the link. $\endgroup$
    – knrumsey
    Commented May 24, 2019 at 19:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.