# Using n-1 or just n in a significance test

Have edited this post quite substantially to make it more focused and more standalone from my previous Excel spreadsheet-focused question.

To give some context, I am trying to assess whether a marketing test has had an impact upon the survival of recurring financial donors. The donors are signed up to give a direct debit each month from their bank account and this particular test has monitored whether an email or a postal mail welcome pack has an impact on survival.

I've been trying various web-based significance calculators and settled on one at http://www.surveystar.com/ztest.htm, as it helpfully shows more detail than some other calculators (namely the z score and 1- and 2-tailed values).

The following formula is taken from the webpage's Javascript:

z = (p1.value - p2.value) /
Math.sqrt((p1.value * (1 - p1.value) / (n1.value - 1))+(p2.value * (1 - p2.value) / (n2.value - 1)))


Essentially my main question is around whether the use of 'n1.value - 1' is appropriate. I am vaguely aware that it's advisable to use n-1 rather than n when working from a sample group rather than the whole population. Does that apply to this context, where I am looking at all of the people in this test? Should they be treated as a sample, or rather treated as the whole population?

Whether I use n or n-1 seems to have a negligible effect on the z scores produced, but I just wanted to make sure I had understood this rather than blindly copying it from the internet!

• Providing access to spreadsheets is fine, and your attention to that detail is appreciated--but your question needs to be understandable without requiring readers to open them. Could you therefore describe what you mean by "compute a z score in this way"?
– whuber
Sep 14, 2017 at 18:30
• Thank you whuber. I have edited the question so it is hopefully now self-contained within this post and not reliant on accessing my spreadsheet. Sep 19, 2017 at 14:54

First off, the correct formula for the z-test comparison two independent proportion uses $n$ and not $n-1$ in the denominator. I'm not sure where the $n-1$ came from in the above but I suspect it migrated from a t-test comparing two independent means. The correct formula is: $$z = \frac{p_1 - p_2}{\sqrt{\frac{p_1(1-p_1)}{n_1}+\frac{p_2(1-p_2)}{n_2}}}$$ With sufficiently large sample sizes, the results will be similar. That said, the n-1 is not appropriate for proportions as the standard error is a known function of the true population value and sample size.

Second, this approach is an approximation given that it is testing a discrete distribution with a continuous one. This can be corrected with a continuity correction or by using Fisher's Exact test. However, with a reasonable sample size, the z-test approximation is sufficiently accurate for most purposes. See for a discussion: H to check if proportions in two small samples are the same .