I've experimentally achieved my goal by running many random trials, generating two points according to scaled and translated gaussian distributions and counting how many times x is less than y.

However this is becoming a problem from a performance point of view for my application.

Is there a straightforward way to compute P(X < Y) where X and Y are variables with a given mean and deviation?

  • $\begingroup$ Are you assuming that $X$ and $Y$ are independent? Are their marginal distributions Gaussian (you mention using Gaussian variables in your "algorithm" but not your actual problem statement)? Are they jointly Gaussian? $\endgroup$
    – Chris Haug
    Aug 15 '17 at 0:10
  • $\begingroup$ I believe them to be independent. I'm not very good with statistics, so I can't directly answer your other two questions. The distributions are being taken from a ranking algorithm called Glicko2 which gives a mean and deviation for a player, and I am trying to determine how much or how little two players may overlap. $\endgroup$
    – TomatoCo
    Aug 15 '17 at 0:32

If your assumptions regarding the normality of $X$ and $Y$ are correct, then you have two normally distributed random variables $X\sim\mathcal{N}(\mu_X, \sigma_X^2)$ and $Y\sim\mathcal{N}(\mu_Y, \sigma_Y^2)$, and you are searching for $\mathrm{Pr}(X < Y)$, aka $\mathrm{Pr}(X-Y < 0)$.

Assuming $X$ and $Y$ are independent, this is easy to analyze, because in that case $X-Y\sim\mathcal{N}(\mu_X-\mu_Y, \sigma^2_X+\sigma^2_Y)$. Therefore your desired probability can be obtained from the standard normal cdf: $\mathrm{Pr}(X-Y < 0) = \Phi(\frac{\mu_Y-\mu_X}{\sqrt{\sigma^2_X+\sigma^2_Y}})$.

To link this to your approach of simulating many samples from the random variables, consider the case where $X\sim\mathcal{N}(1, 3^2)$ and $Y\sim\mathcal{N}(2, 4^2)$. Then we can calculate that $\mathrm{Pr}(X < Y) = \Phi(\frac{2-1}{\sqrt{3^2+4^2}})\approx 0.57926$. We can obtain a similar result via simulation in R:

x.samples <- rnorm(1e6, 1, 3)
y.samples <- rnorm(1e6, 2, 4)
mean(x.samples < y.samples)
# [1] 0.579655

If $X$ and $Y$ are not independent, then we need more information to calculate $\mathrm{Pr}(X < Y)$. For instance, if they are jointly normally distributed with correlation coefficient $\rho$, then $X-Y\sim\mathcal{N}(\mu_X-\mu_Y, \sigma_X^2+\sigma_Y^2+2\rho\sigma_X\sigma_Y)$, meaning $\mathrm{Pr}(X < Y) = \Phi(\frac{\mu_Y-\mu_X}{\sqrt{\sigma_X^2+\sigma_Y^2+2\rho\sigma_X\sigma_Y}})$. If they are not jointly normally distributed, then $X-Y$ may have some other distribution, and that will impact the calculation of $\mathrm{Pr}(X < Y)$.

  • $\begingroup$ Thank you! This works for my purposes. Do you have a preferred method of computing Phi? $\endgroup$
    – TomatoCo
    Aug 15 '17 at 0:28
  • $\begingroup$ @TomatoCo I use the pnorm function in R. For instance, in my answer I calculated using pnorm((2-1)/sqrt(3^2+4^2)). $\endgroup$
    – josliber
    Aug 15 '17 at 0:33
  • $\begingroup$ Unfortunately I'm not using R. I've found an algorithm that differs from my experimental probability by about 3e-5 on average, so I'm pretty happy with that. I was hoping you might know a best practice but it seems for you the best practice is to use a builtin for your language. I'll exercise my google-fu now that I have a good starting point. Thanks again. $\endgroup$
    – TomatoCo
    Aug 15 '17 at 0:41
  • 1
    $\begingroup$ @TomatoCo I do not agree. There has been no testing of the normal assumption here such that nonparametric methods of significant difference of location should be used by default. Alternatively, do the tests first, then decide. $\endgroup$
    – Carl
    Aug 15 '17 at 20:39
  • 3
    $\begingroup$ @Carl testing assumptions before choosing what procedure to use is often distinctly suboptimal (in that the properties of the subsequent options are both impacted, sometimes badly). If the OP assumed normality, it's perfectly valid to answer given that assumption (i.e. to say "if your assumption is correct, ..."). It may be a good idea to suggest the OP consider whether the assumption is close to true, or (better) how sensitive the answer may be to the assumption, but to say that the answer is wrong because no test was conducted may actually lead the OP into poor options. $\endgroup$
    – Glen_b
    Aug 16 '17 at 4:07

Not the answer you're looking for? Browse other questions tagged or ask your own question.