4
$\begingroup$

My situation is I have a list of people that were included in a marketing campaign. Half the list got version 1 of an email and the other half got version 2. Then people either bought the product or didn't.

Lets say both lists had 200 people. And in list 1(version 1 email) 20 bought the product. and in list 2 24 people bought the product.

In absolute terms it seems the version 2 email is more successful. Can someone tell me how to determine statistical significance for this test? and additionally, if you know of any resources where I can read/learn more about the topic of knowing which test to run in which scenario.

Thanks

Improve this question
$\endgroup$
5
  • 2
    $\begingroup$ chi square test? en.wikipedia.org/wiki/Chi-squared_test or just simple logistic regression with 0-1 for the campaign and see if it is significant $\endgroup$
    – rep_ho
    Commented Nov 12, 2019 at 14:53
  • $\begingroup$ I think you are looking into proportion test stat.ethz.ch/R-manual/R-devel/library/stats/html/prop.test.html $\endgroup$
    – German Demidov
    Commented Nov 12, 2019 at 14:54
  • $\begingroup$ prop.test for two proportions, if the correct=FALSE option is chosen, is equivalent to a chi-square test of association (also without continuity correction). $\endgroup$
    – Sal Mangiafico
    Commented Nov 12, 2019 at 15:04
  • $\begingroup$ @rep_ho thank you I will read about that. I understand the basics of logistic regression, how would I apply that for this case? The independent variable would be the the campaign version? Thanks! $\endgroup$
    – Jamalan
    Commented Nov 12, 2019 at 15:10
  • $\begingroup$ There's some description of logistic regression, confidence intervals for proportions, and chi-square test of association for a similar question here. $\endgroup$
    – Sal Mangiafico
    Commented Nov 12, 2019 at 16:06

1 Answer 1

Reset to default
3
$\begingroup$

Comment. Results from a one-sided 'test of two proportions' in Minitab shows no significant difference. (The R procedure prop.test will give a similar result.)

The first P-value is for an approximate normal test. The second is for 'Fisher's exact test', which uses a hypergeometric distribution. (The R procedure fisher.test will give a similar result.)

[Each of the tests is discussed on several pages on this site. Search terms in 'single quotes' above for more discussion. Perhaps start with pages linked above or under 'Related' in the right margin.]

Test and CI for Two Proportions 

Sample   X    N  Sample p
1       20  200  0.100000
2       24  200  0.120000

Difference = p (1) - p (2)
Estimate for difference:  -0.02
95% upper bound for difference:  0.0314395
Test for difference = 0 (vs < 0):  
  Z = -0.64  P-Value = 0.261

Fisher’s exact test: P-Value = 0.316

In R, the hypergeometric CDF function phyper returns Minitab's P-value:

phyper(20, 200,200, 44)
[1] 0.3160336
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.