I've spent several hours online researching an appropriate means of using statistics to determine whether or not variation test results are meaningful, or just chance.
If the testing took the format of an email campaign, where two different sequences of emails were sent to a population of people, what would be an appropriate means of determining success?
As of and up to September 1st 2014, new email subscribers were sent 5 emails within 30 days. A new email campaign was conceived, where 5 emails were sent but with different content. From September 1st, as new subscribers signed up, each alternating address was placed into either the original sequence of emails or the new sequence.
The goal is that after 30 days the subscribers will become customers.
So I now have data that looks like this:
Group | Subscribers | Customers A | 500 | 50 B | 500 | 60
Group | Subscribers | Customers
A | 500 | 50
B | 500 | 60
So the conversion rate for group A is 10% and 12% for group B.
Group B, the new sequence, performs better than group A.
Is there an opportunity to use stats to underline this statement?
I was read lots of posts on Chi Square tests but the thing I couldn't reconcile was "expected". The web is full of examples of voting intentions of men Vs. women in terms of right Vs. left (Republicans and Democrats). In those examples the expected for Null hypothesis was 50/50.
My data is laid out in a way that looks like chi square examples but, since I have no "expected" I am unable to proceed.
Can anyone point me in the right direction here?