# Background

In business industry, I came across two different type of CTR metrics to measure our products. I want to perform hypothesis testing(AB test) for these metrics. However, I am not sure about my test statistics' distribution.

Click-through rate (CTR) is the ratio of users who click on a specific link to the number of total users who view a page, email, or advertisement. It is commonly used to measure the success of an online advertising campaign for a particular website as well as the effectiveness of email campaigns. -- wiki

The general formular of CTR is $$ctr = \frac{\mathrm{the \ number \ of\ click-throughs}}{\mathrm{the \ number\ of\ exposures}}$$. It can be calculated over each sample point or over the whole sample.

# My Question

Let's set a click through rate scenario. Suppose we have only two user on CrossValidated landing page. I opened it 4 times and clicked it 2 time while you opened it 7 times and clicked it 3 times.

• One metric is uPVCTR: Average of each user's PVCTR. Mine PVCTR is $$\frac{2}{4}= 0.5$$ and yours is $$\frac{3}{7}$$. $$\mathrm{uPVCTR}= \frac{\frac{2}{4} + \frac{3}{7}}{2}$$

• The other is overall PVCTR: PVCTR = $$\frac{\sum{clicks}}{\sum{exposure}} = \frac{2+3}{4+7}$$.

Suppose each user is independent, thus each ones' PVCTR is independent. The distribution of uPVCTR is asyptotic normal. If we have control and treatment group, and we denote $$\mathrm{diff} = \mathrm{uPVCTR_{treatment}} - \mathrm{uPVCTR_{control}}$$. We can use $$\frac{diff}{\mathrm{var}(diff)}$$ to perform a two sample T-test.

However, what's the distribution of Overall PVCTR? How to choose a test statistics for it?

• You can clarify the question much by giving the general formulas for the CTR metrics. Now your formulas contain fractions with constants. Also 2-3 sentences context about CTR will give this forum a clearer view on your question. Commented Aug 28, 2020 at 6:48
• Edited, adding explanation for CTR. This is a general statistic problem Commented Aug 28, 2020 at 8:40

It is clear that applying the log function to $$CTR$$ yields a variable which is likely to be normally distributed.

Define

$$\log(CTR)=\log(\# \; click\, throughs) - \log(\# \; exposures)$$

Still, $$\log(CTR)$$ is expected to vary with the hour, and over the weekdays.