4
$\begingroup$

Assuming I have proportions for clicks to impressions of 0.10 for p1 and 0.05 for p2. This is equivalent to 10% and 5% click through rate.

One way is to say that p1 is 5% higher but can we even say that p1 is 100% more than p2 ? If so what is the best possible way to word this to not confuse the reader if only the delta is displayed and not the ratios

Option 1) The first proportion is 5% higher ?

Option 2) There is a 100% delta between both proportions

Edit: Typo in my delta number example (changed it)

$\endgroup$
1
  • $\begingroup$ "P1 is 2-fold larger than P2". This wording is as ambiguous as "twice as large", but works for any decimal values rather than just round numbers like 2. $\endgroup$
    – bjb568
    May 26 at 9:32

3 Answers 3

12
$\begingroup$

As stated by Dave, $p_1$ is $100\%$ more than $p_2$, not $50\%$ (now corrected in the question), and is equivalent to saying that it is twice as much.
As for the other way of saying it (option 1), I believe that the convention generally is to say that $p_1$ is 5 percentage-points higher than $p_2$.

$\endgroup$
5
$\begingroup$

$p_1$ is not $50\%$ higher than $p_2$; $p_1$ is $100\%$ higher than $p_2$.

I would phrase it accordingly: $p_1$ is double $p_2$. After all, I would expect the event under consideration to happen twice as often under the conditions giving $p_1$ as under the conditions giving $p_2$.

In other words, if you make a dollar every time the event happens, you make twice as much money with $p_1$ as with $p_2$.

$\endgroup$
1
  • $\begingroup$ assuming one can only show percentages and not words as such "double ..." this is in the context of ab testing $\endgroup$ May 25 at 13:46
4
$\begingroup$

There isn't really a single, 'correct' way to describe the relationship between two proportions. The issue here is how to give a measure of the size of an effect when the constituent measures are proportions of known totals. The effect size for comparisons of continuous variables with natural zeros is straightforward, but with binary data, there are inescapable complexities. Unfortunately, there is no perfect solution to that problem whereby you get all the advantages of every option and none of the disadvantages. Moreover, which measure of effect size should be used for such a situation has long been a source of contention. For what it's worth, the three major contenders are:

  1. The risk difference (your option 1), which is $.05$ in your case. People like it because subtraction is simple and the measure seems very intuitive. A problem is that, in general, $.05$ probably does not equal $.05$ (e.g., it is likely much harder to improve your click-through rate from $.10$ to $.15$ than it is to improve it to $.10$ from $.05$, as the pool of potential customers is shrinking within the group that sees your ad).

  2. The risk ratio (more or less your option 2), which is $2$ in your case. People similarly like this as relatively intuitive. Note however that successive increments with the same numerical value are again unlikely to be truly equal (similar to the above; here: $.1/.05 \ne .2/.1$), and that the measure is unintuitively asymmetrical (i.e., comparing $p1$ to $p2$ yields $2$, but comparing $p2$ to $p1$ gives $.5$).

  3. The odds ratio (which you did not list), which would be $\approx 2.11$ in your case. With the odds ratio, it is at least theoretically possible to have a constant increment forever, and the odds ratio also plays an important role in logistic regression. On the other hand, the odds ratio isn't even remotely intuitive, and suffers from a technical problem called 'non-collapsibility'.

In your case, you have just two numbers. It is probably easiest to just present them. If you have to say it's either 'five percentage points higher' or 'twice as high', I would focus on which is likely to be more readily and correctly understood by your audience. I might further consider a case where you were able to be twice as effective, would the result then be closer to $.15$ or $.20$?

$\endgroup$
2
  • $\begingroup$ love the answer. I just think that in the context of ab testing specifically in cases where we're evaluting click through rates. if control is 20% while test is 15%, i dont see the logic in doing the relative % difference when the actual true change is simply 5%. I think that in this case , presenting the difference in %s is leading to more confusion than not ? $\endgroup$ May 26 at 23:01
  • $\begingroup$ @RogerSteinberg, that's fine. It's a judgement call for you to make based on your context & your audience. $\endgroup$ May 27 at 11:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.