Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 0 down vote favorite
1

Here is an explanation of how NPS is calculated:

http://en.wikipedia.org/wiki/Net_promoter_score

I'm interested in testing two net promoter scores to determine if they are statistically different. I read a great answer to calculating margin of error for NPS (see link below), but I'm really interested in testing to see if there is a difference between two scores, because I suspect that our results aren't as "different" from year to year as they appear to be.

How can I calculate margin of error in a NPS (Net Promoter Score) result?

Is this at all possible? I understand t-tests are typically used to test whether two different record sets are statistically different. But is it possible to test Net Promoter Scores, either with a t-test or some other hypothesis test?

Any ideas you have would be a great help. Thank you!

share|improve this question
2  
I think the answer to your question is embedded in @whuber's excellent answer that you have linked to. He effectively suggests what is sometimes called a Z test, but with samples of above 50 or so this will be the same as a t test. NPS will be approximately normally distributed, and you can use his methods of calculating standard errors. – Peter Ellis Mar 2 '12 at 20:13
Thank you for your reply Peter. I'll give that a try. – Charles Naughton Mar 2 '12 at 23:18

1 Answer

up vote 1 down vote

Whuber's answer is excellent indeed. Another way, which is more straightforward/simple is to :

  1. Calculate the confidence interval of the promoters and for the detractors. Let us denote a P for promoters and D for detractors

  2. Then calculate : (P + error of p) - (D - error of d) for the highest end of NPS confidence interval and (P - error of p) - (D + error of d) for lowest end of the NPS confidence interval

In other words, to get NPS confidence interval, just add the error of promoters and the error of detractors and NPS +/- sum of both errors will be your confidence interval.

Now after you have the confidence interval of your 1st NPS result and the confidence interval of your 2nd NPS result, compare these two to assess if there is a significant difference.

share|improve this answer
I appreciate you taking the time to reply Sheriff, but I'm not sure that what you're saying is correct. I don't believe that the error calculated on the proportion of promoters added to the error on the proportion of detractors is equal to the error of NPS (which is essentially what you are saying in your example). Each of the respective proportions is not independent of one another. The proportion of the detractors impacts the proportion of promoters, and vice versa. The proportions aren't independent, and neither are the errors. – Charles Naughton Dec 6 '12 at 22:22
I believe the best answer was provided in the post I referenced above, which calculates the error, which you can use to calculate the upper and lower limits of the confidence intervals for the NPS. Also, confidence intervals aren't enough to determine if two Net Promoter Scores are statistically different. To explain further, the NPS of your sample is essentially an estimate of the NPS of your population. You can use confidence intervals to help you determine how reliable that estimate is. – Charles Naughton Dec 6 '12 at 22:22
Confidence intervals are not really used to determine the whether two separate estimates are statistically different or not. However, a statistical hypothesis test such as a z-test, can tell you within a certain probability whether or not two NPSs are different. I don't mean to "correct" you, so please don't take offense. I just wanted to make sure people who stumble on this thread are aware of all the considerations. – Charles Naughton Dec 6 '12 at 22:22

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.