Background
In business industry, I came across two different type of CTR metrics to measure our products. I want to perform hypothesis testing(AB test) for these metrics. However, I am not sure about my test statistics' distribution.
Click-through rate (CTR) is the ratio of users who click on a specific link to the number of total users who view a page, email, or advertisement. It is commonly used to measure the success of an online advertising campaign for a particular website as well as the effectiveness of email campaigns. -- wiki
The general formular of CTR is $ctr = \frac{\mathrm{the \ number \ of\ click-throughs}}{\mathrm{the \ number\ of\ exposures}}$. It can be calculated over each sample point or over the whole sample.
My Question
Let's set a click through rate scenario. Suppose we have only two user on CrossValidated landing page. I opened it 4 times and clicked it 2 time while you opened it 7 times and clicked it 3 times.
One metric is uPVCTR: Average of each user's PVCTR. Mine PVCTR is $\frac{2}{4}= 0.5 $ and yours is $\frac{3}{7}$. $\mathrm{uPVCTR}= \frac{\frac{2}{4} + \frac{3}{7}}{2}$
The other is overall PVCTR: PVCTR = $\frac{\sum{clicks}}{\sum{exposure}} = \frac{2+3}{4+7}$.
Suppose each user is independent, thus each ones' PVCTR is independent. The distribution of uPVCTR is asyptotic normal. If we have control and treatment group, and we denote $\mathrm{diff} = \mathrm{uPVCTR_{treatment}} - \mathrm{uPVCTR_{control}}$. We can use $\frac{diff}{\mathrm{var}(diff)}$ to perform a two sample T-test.
However, what's the distribution of Overall PVCTR? How to choose a test statistics for it?