0
$\begingroup$

So, I will run an email campaign and I want to calculate the difference between the current NPS score and the one after the campaign, to see if the campaign is actually making any impact at all. By the way, the sample sizes are going to be different. So my question is, what statistic calculation can I use for this?

Additional details, the score is ranked from 1 to 10. The sample sizes are different sizes, around 30 and 100. The hypothesis is that the testing of the sample averages are statistically different.

$\endgroup$
1

1 Answer 1

1
$\begingroup$

The strongest answer you might get is if you are able to pair the before/after comparisons by each individual surveyed and examine the specific "difference" to see whether it is non-zero. If not, I guess the standard text book answer to this question is to use the Mann-Whitney / Wilcoxon test: e.g. https://stats.idre.ucla.edu/spss/whatstat/what-statistical-analysis-should-i-usestatistical-analyses-using-spss/ A few caveats: (a) The null hypothesis would state that the population medians of the before/after group are identical, you are hoping to reject that hypothesis based on the evidence provided by the samples (b) Your sample sizes are small and you may find yourself unable to reject the null hypothesis, even if there is a difference (c) More important than the statistical inference, any before/after study runs the risk that a lot of other things in the real world not related to your campaign that may have altered the NPS score (d) there are some technical assumptions you are making e.g. that the individuals polled are a random sample, and the responses are independent of each other. There are more complex methods available which help deal with any problematic assumptions..

$\endgroup$
1
  • $\begingroup$ There will be a huge number of ties to account for in the MW test, suggesting one should have reservations about its application. Of course in the situation presented in this question a pairing is impossible: the sample sizes are different. $\endgroup$
    – whuber
    Commented Sep 20, 2019 at 14:09

Not the answer you're looking for? Browse other questions tagged or ask your own question.