I have two groups A and B. Each group involves some number of users. For example, let's imagine that the group A has 2000 unique users on Monday, while the group B has 2002 users on the same day.
For each group I measure the number of clicks that users make on a web site. For example, let's say that the users of group A made (totally) 400 clicks (the same user can make more than one click), while the users of group B made 450 clicks on the same page.
Then for each group I calculate the performance metric (in %) as follows: 100 * number of clicks / unique number of users.
PerfA PerfB NumUsersA NumUsersB 20% 22% 2000 2002 21% 20% 2001 1999
For each pair of values PerfA and PerfB, I want to calculate the statistical significance of a difference.
Which statistical test should I use? If the Python/Scala library can be suggested, I would be very happy.
I found some examples of how the statistical significance is measured in Excel, but I am now sure about the statistical test (formula) that it refers to: