Testing for interaction between two categorical independent variables and a dependent variable

Question

This is a contrived example for clarity, but it reflects a real problem that I'm dealing with:

Say that I want to analyse miles-per-gallon (ratio) for cars of different color and interior type (both categorical):

colors: red, green, blue

interior types: leather, cotton, synthetic

I don't necessarily compare the same number of cars in each category:

Scenario 1:
+-------+----------+----------+-------------+
| MPGs  | Leather  |  Cotton  |   Synthetic |
+-------+----------+----------+-------------+
| Red   | 10,11,12 | 11,12,13 |       13,14 |
| Green |       10 | 10,11,12 |    11,12,13 |
| Blue  |   8,9,10 |  9,10,11 |    10,11,12 |
+-------+----------+----------+-------------+ 

In the above example, it's seems that red has higher mpg than green or blue, independent of interior style, and that synthetic has higher mpg than leather or cotton, independent of color.

Scenario 2:
+-------+----------+----------+-------------+
| MPGs  | Leather  |  Cotton  |   Synthetic |
+-------+----------+----------+-------------+
| Red   | 10,11,12 | 11,12,13 |       13,14 |
| Green |       10 | 11,12,13 |    10,11,12 |
| Blue  |   8,9,10 |  9,10,11 |    10,11,12 |
+-------+----------+----------+-------------+

The above is similar to before, except that now, if the car is green, then cotton seems to have the highest mpg.

Basically, I need to find out if I can look for the best interior type independently of the car's color. My motivation for asking this is that if I can ignore color, then each interior type has about 9 observations, thus I can be more certain which is best. But if I can't, then each [color,interior type] group has only about 3 observations, which means I'm less confident about which is best.

So I have two questions here:

1. What am I doing here? Is this really a test for interaction?

2. What statistical test can be used here?

Thanks!

Answer 1

I'm going to provide a Bayesian answer, I suspect/hope others will provide a more frequentist view.

I would first come up with an answer to the question "given a car which I know nothing about, what would be the distribution reflecting my beliefs about the likely mpg of that car, given the data set I've just seen?". As a Bayesian, you have to make some assumptions, perhaps the mpg distribution of all cars is Gaussian or (as it technically can't be normal) gamma, lognormal etc. Using this plus a prior, calculate a posterior distribution. Effectively you're saying, given the cars you've seen, what do you think the range of possible mpgs of all cars out there might look like.

You can perform the same for cars of all colours, and for cars of all interiors. Finally, you can do the same for cars of all colour-interior combinations. Of course, as you go more and more sparse, you have less data, meaning your likelihood function will be less peaked and your posterior will look more like your prior (which, provided you chose a sensible prior, will be pretty wide, as you should approach the problem with an open mind, but this subjectively is clearly the weakness of any Bayesian method).

Based on your posteriors, you can start to answer questions like "what is the probability that a red car has an mpg of at least 1 better than a car I have no information about?", or "what is the probability that a red car has an mpg of at least 2 better than a green car?"

As you start asking more detailed questions such as "what is the probability that a red, interior leather car has an mpg of X better than a green car with a cotton interior?", you'll start to find that you get less and less useful answers (e.g. the probability is basically the same as if you had no information about the cars), so you reach this tradeoff where having more information about the car is actually like having less, because you have less data to base your assertions on.

Yes, this approach has subjective elements, the choice of distribution for your likelihood (technically you can do model selection, but this is hard work), and the prior, and requires a lot of manual examination of plots and judgements, but in many ways it's also very intuitive and in my own personal opinion, very powerful.