I am running a test in Google Analytics to find the impact of a new functionality release for sharing stuff.

The setup follows: - There is a control group which continues to see and use the old "sharing" functionality - There is an exposed group which sees and uses the new "sharing" functionality"

I am trying to identify whether the new functionality performs better.

I believe I am trying to answer 2 questions:

  1. Do more people share with the new functionality?
  2. Out of those who do share, has the average quantity of shares increased?

For the first question I have run a Chi-Squared test and got the following results: http://www.evanmiller.org/ab-testing/chi-squared.html#!1851/21926;46/926@95

Basically there were 21926 users in control and 1851 shared something (one or more shares), and there were 926 users exposed and 46 shared something (one or more shares).

My question are:

  1. Am I setting up this test up correctly and using the right statistical methodology?
  2. What statistical methodology should I use to measure the result for question 2? Assuming that for the control group the 1851 users shared 4321 items, and for the exposed group the 46 users should 133 items.

Many thanks.

  • $\begingroup$ I think you can use t-test (if the assumptions are met) for both objectives. $\endgroup$ – Dr Nisha Arora Mar 21 '16 at 11:31

This is a good start, but it is not set up correctly. Instead of having the # of trials, you need the # of FAILS which is trials-successes.

| cite | improve this answer | |
  • 2
    $\begingroup$ This is being automatically flagged as low quality, probably because it is so short. At present it is more of a comment than an answer by our standards. Can you expand on it? We can also turn it into a comment. $\endgroup$ – gung - Reinstate Monica Jan 9 '17 at 18:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.