1
$\begingroup$

Have edited this post quite substantially to make it more focused and more standalone from my previous Excel spreadsheet-focused question.

To give some context, I am trying to assess whether a marketing test has had an impact upon the survival of recurring financial donors. The donors are signed up to give a direct debit each month from their bank account and this particular test has monitored whether an email or a postal mail welcome pack has an impact on survival.

I've been trying various web-based significance calculators and settled on one at http://www.surveystar.com/ztest.htm, as it helpfully shows more detail than some other calculators (namely the z score and 1- and 2-tailed values).

The following formula is taken from the webpage's Javascript:

z = (p1.value - p2.value) / 
    Math.sqrt((p1.value * (1 - p1.value) / (n1.value - 1))+(p2.value * (1 - p2.value) / (n2.value - 1)))

Essentially my main question is around whether the use of 'n1.value - 1' is appropriate. I am vaguely aware that it's advisable to use n-1 rather than n when working from a sample group rather than the whole population. Does that apply to this context, where I am looking at all of the people in this test? Should they be treated as a sample, or rather treated as the whole population?

Whether I use n or n-1 seems to have a negligible effect on the z scores produced, but I just wanted to make sure I had understood this rather than blindly copying it from the internet!

$\endgroup$
2
  • $\begingroup$ Providing access to spreadsheets is fine, and your attention to that detail is appreciated--but your question needs to be understandable without requiring readers to open them. Could you therefore describe what you mean by "compute a z score in this way"? $\endgroup$
    – whuber
    Sep 14, 2017 at 18:30
  • $\begingroup$ Thank you whuber. I have edited the question so it is hopefully now self-contained within this post and not reliant on accessing my spreadsheet. $\endgroup$ Sep 19, 2017 at 14:54

1 Answer 1

0
$\begingroup$

First off, the correct formula for the z-test comparison two independent proportion uses $n$ and not $n-1$ in the denominator. I'm not sure where the $n-1$ came from in the above but I suspect it migrated from a t-test comparing two independent means. The correct formula is: $$ z = \frac{p_1 - p_2}{\sqrt{\frac{p_1(1-p_1)}{n_1}+\frac{p_2(1-p_2)}{n_2}}} $$ With sufficiently large sample sizes, the results will be similar. That said, the n-1 is not appropriate for proportions as the standard error is a known function of the true population value and sample size.

Second, this approach is an approximation given that it is testing a discrete distribution with a continuous one. This can be corrected with a continuity correction or by using Fisher's Exact test. However, with a reasonable sample size, the z-test approximation is sufficiently accurate for most purposes. See for a discussion: H to check if proportions in two small samples are the same .

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.