I've run a test with one control and one experiment group, and am questioning myself on whether or not I've used the right test (or if significance can even be calculated on the following sample sizes).
The data is as follows:
The control cohort (A) had 63 people see the treatment and 1 person performed the action (1.59%)
The Experiment cohort (B) had 64 people see the treatment and 9 people performed the action (14.1%)
I used a z-test for two population proportions (this equation: http://www.socscistatistics.com/tests/ztest/) to compare the two proportions. It says that the number of people who performed the action in B is a statistically significant increase over the number of people who performed the action in A with a p-value of 0.00453.
However I wanted to make sure that:
- a) I'm using the right test -- I know t-tests are sometimes better tests to use when samples sizes are small
- b) Statistical significance can even be determined on such a small sample.