1
$\begingroup$

I am new to statistics, but I need to see if results of my findings are significant, but I’m not sure, which t test to use in my case.

I have a product page, that may have from 1 to ~10 videos. So, I need to compare CTR(Click Through Rate - number of users who clicks on my video to number of users who saw this page with video).

I need to calculate, if there is significant difference in CTR depending on video is alone on he page or video is located together with other videos.

Let’s say I have 7 videos and I want to compare where those videos performed better (where CTR is better) : when they are alone on page or when they are together with other videos.

So my input for t test is looks like this:

enter image description here

where column TRUE - means that this videos is alone on page vs FALSE - video is not alone. CTR is in %

From this table, I can see that all videos are performing better (meaning that has better CTR) when they are alone on page. But I need to confirm that this results are statistically significant.

So my question is which t-test I need to use to confirm this? As I understand, I cannot use independent sample T-test, since I am comparing same videos, but in different condition?

Improve this question
$\endgroup$
1
  • $\begingroup$ Can you post a histogram of your data? It looks like they might be skewed (see here: stats.stackexchange.com/questions/111320/…) so I'm not sure any sort of t-test is really appropriate. You might want to considered the Wilcoxon signed rank test as an alternative. $\endgroup$
    – TPM
    Commented May 14, 2018 at 16:20

1 Answer 1

Reset to default
0
$\begingroup$

you should use a paired sample t-test.

one way to implement it is:

  1. for each video you compute the difference TRUE-FALSE
  2. you apply a 1-sample t-test to the so obtained array: if it is significantly bigger than zero, then CTR is significantly better when the video is alone
Improve this answer
$\endgroup$
4
  • $\begingroup$ I am trying to use online tool to get my p value (this one: socscistatistics.com/tests/ttestdependent/Default2.aspx) There is a checkbox for One-tailed or two-tailed hypothesis? Would my hypothesis considered to be one-tailed? $\endgroup$
    – Bilberryfm
    Commented May 14, 2018 at 17:05
  • $\begingroup$ So I tried 2 t tests: - One called "T-Test Calculator for 2 Dependent Means " (socscistatistics.com/tests/ttestdependent/Default2.aspx) - Second called: "Single Sample T-Test Calculator" (only for TRUE-FALSE diff, as you suggested) socscistatistics.com/tests/tsinglesample/Default2.aspx. And I got completely diff results: for first t test results are statistically significant (p is 0.030506). For second one - it's not (P-Value is 0.5) $\endgroup$
    – Bilberryfm
    Commented May 14, 2018 at 17:11
  • $\begingroup$ whether you use a one tail or two tail is not a statistics question, it depends on your expectations as a researcher. for example if you have a theory saying that many videos work as a distractor and as a consequence you expect they reduce the CTR, then fine to use a 1 side. if you are just planning a new website and you have no idea which is better, or if one of the two possibilities is better at all, then you should use 2 sided. $\endgroup$
    – fabiob
    Commented May 15, 2018 at 19:42
  • $\begingroup$ on the other hand for me the two tests you used give the same result (both for 1-tailed and 2-tailed). I first computed "T-Test Calculator for 2 Dependent Means", which then also provides a column with the difference. this column (Diff(T2-T1)) I used as an input for "Single Sample T-Test Calculator", where I also had to set population mean =0. $\endgroup$
    – fabiob
    Commented May 15, 2018 at 19:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.