Our company sells a product "A" to different region of the world. Our department measure the reliability of the product by measuring the % of product that failed and return by our customers.

In each regions there are a several customers, but the same customer can't be in 2 regions. We measure customers satisfaction rate (= % reliable product) by year, within each region, and then average customers in the same year and region, so we get a total satisfaction rate / year / region.


We want to assess if the average reliability of the product "A" is the same across regions or if some region are significantly different.


The proposed approach was to use an ANOVA the data below

enter image description here


  • Is ANOVA a valid test to compare the mean of both this region ?
  • If the test is positive (and we reject the null hypothesis: performance Region1 = Region2), is it reasonable to assume there is some causal relationship between the region and the performance?


To clarify the question, the goal is not to estimate the reliability of each region but to assess if the difference between the average reliability in each region is statistically significant or not.

  • 1
    $\begingroup$ Correlation is definitely an indicator of "some causal relationship" - but it cannot by itself determine what that relationship looks like, or what other variables may be involved, hence the credo "correlation does not imply causation". $\endgroup$
    – Scriddie
    Jan 11 at 20:49
  • 2
    $\begingroup$ You lost useful information when you normalized the counts (number x returns out of n orders) into %'s (esp. if the number of orders vary by region.) $\endgroup$
    – dipetkov
    Jan 11 at 21:45
  • $\begingroup$ @dipetkov I agree, and in practice this is the case, especially between customers. Some place bigger orders. The idea of normalizing was to avoid one big customer in a region skewing the number for the entire region. Our goal is to understand the impact of region on the product reliability, independent of a customer individual behavior. I thought averaging the rate would help. Or am I misguided ? $\endgroup$ Jan 12 at 14:48
  • 1
    $\begingroup$ I'm not sure what's the best way to deal with one big customer placing a very large order and then returning it; depends on the details perhaps. What I was thinking about is that each number of the 10x2 table is estimated with a different precision / has different standard error. (The standard error of a proportion estimate depends on the sample size n.) The ANOVA is not aware of this. $\endgroup$
    – dipetkov
    Jan 12 at 15:03
  • 1
    $\begingroup$ Updating the question can help. You could also consider writing a new question. Right now it seems you're trying to solve <X> but asked if <Y> is a good way to go about it. If your real interest is <X>, then the current question appears to be an XY Problem. Be as precise about the actual situation as possible. Also, think about whether you want to assume that the rate of return remains constant throughout the years and across the regions, or it's enough to look at whether the rate is the same for both regions even if it might vary from year to year. $\endgroup$
    – dipetkov
    Jan 15 at 20:29

1 Answer 1

  1. If you only have 2 regions you want to compare, a t-test will do just fine (when assumptions are verified, otherwise look into Wilcoxon or Mann-Whitney U tests).

  2. No t-tests and ANOVA's are not used to get evidence about causal effects.

  • $\begingroup$ Thanks, aren't t-test and ANOVA equivalent when comparing two groups ? Not that t-test does not work. We are interested mainly in the lower performance of 1 region, so the problem could be formulated as Region A vs Rest of the World. $\endgroup$ Jan 12 at 14:51
  • 1
    $\begingroup$ yes indeed, it is the same :) It just can cause some confusion since ANOVA is actually used for more than 2 groups, but it is indeed right what you say! $\endgroup$ Jan 12 at 18:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.