0
$\begingroup$

Say you are selling a product, and you know from experience that the green version of the product sells better than the blue version. But you have two types of customer A and B, and you want to know if color has a greater effect on A than it does on B. So you sell the green product in some stores and the blue product in others, keeping track of sales to A and B. Here's the data:

  | Green | Blue
-----------------
A |  25   |  10
-----------------
B |  60   |  50

The effect is clearly a bit larger for group A than it is for group B. But is the difference statistically significant?

My naive idea would be to use an independence test like $\chi^2$ (or maybe Fisher's exact test since the volumes are small). These tests work by comparing the observed data to what is "expected" given the class totals; for instance, in the table above we note that $\frac{35}{145}$ of the customers are of type A and $\frac{85}{145}$ of the products purchased are green, so if color and customer type are independent then we "expect" that $\frac{35 \cdot 85}{145^2}$ of the purchases will consist of a customer of type A buying a green product.

My problem is that this feels like the wrong random model for this situation. Our prior is that green is more popular than blue, and so the contingency table

  | Green | Blue
-----------------
A |  26   |  10
-----------------
B |  61   |  50

might be more likely than

  | Green | Blue
-----------------
A |  26   |  10
-----------------
B |  59   |  50

even though it is "more extreme" from the point of view of our probabilistic calculations under the null hypothesis. So my questions are:

Is there a way to incorporate priors about the distribution of rows / columns in independence tests?

Is independence testing even the right framework for analyzing this experiment?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.