0
$\begingroup$

I want to determine which number of page visits is needed to give a statistical signification of a clickthrough rate of a page.

Example:

Page Visits =100 Clicks On Page= 50 Page Click Through Rate = 50%

How can I find out, how significant the result of 50% is? And how do I define the minimum amount of Page Visits needed for a certain P-value?

$\endgroup$
4
  • 2
    $\begingroup$ To determine significance, you need to have a null model that specifies what happens when there is no effect. What "page click through rate" do you expect when there is no effect, and what do you expect when you make an intervention? $\endgroup$
    – fm361
    Oct 19, 2022 at 12:51
  • $\begingroup$ Thanks for the comment. When the there is no effect, "click on page" = 0 - that said, when a page view is initiated, there are 2 possible outcomes: the user does click, or the user doesn't click $\endgroup$ Oct 19, 2022 at 13:01
  • 3
    $\begingroup$ The word "significant" often gets in the way of clear thinking, especially for people new to statistical thinking. That is because it has too meanings (small p value; and big enough to matter). Try to word your question precisely without using that word, and you might make some progress here. $\endgroup$ Oct 19, 2022 at 13:05
  • $\begingroup$ thanks @HarveyMotulsky - To phrase it without the word significance. How many "Page Visits" are needed, in order to be able to get a reliable indication of the "Page Click Through Rate"? $\endgroup$ Oct 19, 2022 at 13:16

1 Answer 1

1
$\begingroup$

To answer How many "Page Visits" are needed, in order to be able to get a reliable indication of the "Page Click Through Rate"?, as stated in the comments, you might want to try looking at confidence intervals.

You have $N$ page visits and $n$ clicks, and are interested in the page click-through rate $p$.

You could model this as $n \sim \text{Binomial}(N, p)$. Define "reliable estimate of $p$" as a certain interval around $\hat{p}$ (cfr. Binomial_proportion_confidence_interval). Assuming you have a large $N$ and $p$ is not close to 0 or 1, you can get an approximate $100(1 - \alpha)\%$ confidence interval with

$$ \hat{p} \pm z_{\frac{\alpha}{2}} \sqrt{\frac{\hat{p}(1 - \hat{p})}{N}} $$

where $\hat{p} = \frac{n}{N}$ and $z_\frac{\alpha}{2}$ is the $1 - \frac{\alpha}{2}$ quantile of a standard normal (qnorm(1 - alpha/2) in R). Set $\alpha = 0.05$ for a 95% confidence interval.

Going back to the original question, you can compute the size of these intervals on a grid of $p$ and $N$ values to identify an $N$ (that will work for most $p$) that gives you small enough intervals. How small, you should decide.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.