How to qualitatively understand interaction terms?

Question

After many days of deliberation and looking up other similar posts I still feel that I'm missing something in my interpretation of interaction terms in regression. I want to double check to see if my qualitative interpretation is correct for interaction terms concerning cases including two dummy variables, the case for one dummy and one continuous variable situation and the case for two continuous variables.

Two dummy variables case:

Wage = $β_1Female+β_2Black+β_3female*black$

The interaction term $β_3female*black$ is testing two additional differences:

1.We are testing if there's a difference in wage compare to a black female vs a black male. (A hypothesis test of the interaction term in this case is testing if the value $\frac{dtWage}{dtFemale}=β_1+β_3Black$ is significantly different from 0?)

2.We can also test if there's a difference in wage compare to a black female vs non-black female (A hypothesis test of the interaction term is testing if the value $\frac{dtWage}{dtBlack}=β_2+β_3Female$ is significantly different from 0?)

One dummy variable and one continuous variable:

Assuming *calcium * is continuous calcium

$Height = β_1Female+β_2Calsium+β_3Female*Calcium $

In this case is it qualitatively correct to say that there are two ways to explain what $β_3Female*Calcium$ does:

1.We are looking at if there's a difference in height between females who received different levels of calcium.?

2.We are also looking at if there's a difference in height between male or female who received the same level of calcium?

Both variables are continuous case:

$Salary = β_1IQ+β_2Height+β_3IQ*Height$

Again is it correct to say that the interaction term $β_3IQ*Height$ is explaining:

1.We are looking at if there's a difference in Salary given a level of IQ at different levels of height?

2.We are looking at if there's a difference in salary given a level of height at different levels of IQ?

Many thanks in advance!!

Answer 1

Two dummy variable case

I think you would find it easier if you expressed your first model like this:

Wage = $β_0 + β_1Female + β_2Black + β_3Female*Black + \epsilon$

If the error term $\epsilon$ has a mean of zero, that implies that:

Mean Wage = $β_0 + β_1Female + β_2Black + β_3Female*Black$.

Assume the dummy variable $Female$ is defined as: $Female = 1$ if the study subject is a female and 0 if the study subject is a male. Furthermore, assume the dummy variable $Black$ is defined as: $Black = 1$ if the study subject is Black and 0 if the study subject is White. Then the mean wage can be expressed as follows:

Gender = Male; Race = White: Mean Wage = $\beta_0$ (since $Female = 0$ and $Black = 0$);

Gender = Male; Race = Black: Mean Wage = $\beta_0 + \beta_2$ (since $Female = 0$ and $Black = 1$);

Gender = Female; Race = White: Mean Wage = $\beta_0 + \beta_1$ (since $Female = 1$ and $Black = 0$);

Gender = Female; Race = Black: Mean Wage = $\beta_0 + \beta_1 + \beta_2 + \beta_3$ (since $Female = 1$ and $Black = 1$).

Now, simple arithmetic shows that:

a) $\beta_1$ = $(\beta_0 + \beta_1) - \beta_0$ is the difference in mean wage between females and males having white race. In other words, $\beta_1$ is the average wage differential between genders for whites.

b) $\beta_2$ = $(\beta_0 + \beta_2) - \beta_0$ is the difference in mean wage between blacks and whites having male as their gender. In other words, $\beta_2$ is the average wage differential between races for males.

c) $\beta_2 + \beta_3$ = $(\beta_0 + \beta_1 + \beta_2 + \beta_3)$ - $(\beta_0 + \beta_1)$ is the difference in mean wage between blacks and whites having female as their gender. In other words, $\beta_2 + \beta_3$ is the average wage differential between races for females.

Subtracting b) from c) yields that $\beta_3$= ($\beta_2 + \beta_3$) - $\beta_2$ is the difference between (i) the difference in mean wage between blacks and whites having female as their gender and (ii) the difference in mean wage between blacks and whites having male as their gender. So $\beta_3$ is a difference of differences. It contrasts the average wage differential between races across the two genders. If you test the hypotheses:

Ho: $\beta_3 = 0$ versus Ha: $\beta_3 \neq 0$

then you are really testing:

Ho: there is no difference in the average wage differential between races across the two genders
Ha: there is a difference in the average wage differential between races across the two genders

Of course, like you noted, interactions are symmetric, so you could re-express $\beta_3$ as the difference in the average wage differential between genders across the two races and test the corresponding hypotheses.

$$ $$

In your comments, you ask: For the 2nd statement, I was trying to say that given that genders are the same is there a difference in wage between different races. Do you think my approach is also acceptable?.

$$ $$

My answer is that, once you formulated your 4 submodels for mean wage for the 4 possible combinations of gender and race, you can test any hypotheses that are of interest to you. However, you need to make sure you formulate your hypotheses appropriately as a function of the model coefficients. If you want to set up hypotheses to address the research question "among males, is there a difference in mean wage between different races?", start by comparing the following submodels:

Gender = Male; Race = White: Mean Wage = $\beta_0$ (since $Female = 0$ and $Black = 0$);

Gender = Male; Race = Black: Mean Wage = $\beta_0 + \beta_2$ (since $Female = 0$ and $Black = 1$);

This will make it clear that you can address your research question by testing the following null and alternative hypotheses:

Ho: $\beta_2 = 0$ (among males, there is no difference in mean age between blacks and whites)

Ha: $\beta_2 \neq 0$ (among males, there is a difference in mean age between blacks and whites)

$$ $$

If you want to set up hypotheses to address the research question "among females, is there a difference in mean wage between different races?", start by comparing the following submodels:

Gender = Female; Race = White: Mean Wage = $\beta_0 + \beta_1$ (since $Female = 1$ and $Black = 0$);

Gender = Female; Race = Black: Mean Wage = $\beta_0 + \beta_1 + \beta_2 + \beta_3$ (since $Female = 1$ and $Black = 1$).

This comparison will reveal that you can set up your hypotheses as follows:

Ho: $\beta_2 + \beta_3 = 0$ (among females, there is no difference in mean wage between blacks and whites)

Ha: $\beta_2 + \beta_3 \neq 0$ (among females, there is a difference in mean wage between blacks and whites)

$$ $$

If you wish to address the research question "among people of the same gender (be it male or female), is there a difference in mean wage between blacks and whites, then you need to simultaneously test the following sets of hypotheses:

Ho1: $\beta_2 = 0$ (among males, there is no difference in mean wage between blacks and whites)

Ha1: $\beta_2 \neq 0$ (among males, there is a difference in mean wage between blacks and whites)

and

Ho2: $\beta_2 + \beta_3 = 0$ (among females, there is no difference in mean wage between blacks and whites)

Ha2: $\beta_2 + \beta_3 \neq 0$ (among females, there is a difference in mean wage between blacks and whites)

You will need some correction of the resulting p-values for multiplicity, since you are simultaneously performing two tests.

$$ $$ To answer the research question "among black people, is there a difference in mean wage between males and females", you would start by comparing the following submodels:

Gender = Male; Race = Black: Mean Wage = $\beta_0 + \beta_2$ (since $Female = 0$ and $Black = 1$);

Gender = Female; Race = Black: Mean Wage = $\beta_0 + \beta_1 + \beta_2 + \beta_3$ (since $Female = 1$ and $Black = 1$).

This suggests that you need to set up your hypotheses as:

Ho: $\beta_1 + \beta_3 = 0$ (among black people, there is no difference in mean wage between females and males)

Ha: $\beta_1 + \beta_3 \neq 0$ (among black people, there is a difference in mean wage between females and males)

$$ $$ One dummy variable and one continuous variable:

Again, best to write your model like:

Height = $β_0 + β_1Female + β_2Calcium + β_3Female*Calcium + \epsilon$, which will imply that:

Mean Height = $β_0 + β_1Female + β_2Calcium + β_3Female*Calcium$ provided the mean of $\epsilon$ equals 0.

This model shows that the effect of Calcium on Mean Height depends on Gender, since:

Mean Height = $β_0 + β_1Female + (β_2 + β_3Female)Calcium$.

This last model is a collection of two submodels:

Males: Mean Height = $β_0 + β_2*Calcium$;

Females: Mean Height = $β_0 + β_1 + (β_2 + β_3)Calcium$.

Since $β_2$ represents the rate of change in mean height for each extra unit of Calcium in males and $β_2 + \beta_3$ represents the rate of change in mean height for for each extra unit of Calcium in females, the difference between them, $\beta_3$, represents a difference in two rates of change.

Answer 2

I think the reason you are confused is that you are trying to explain the meaning of the interaction term in conjunction with the two individual terms. If you consider the interaction term in isolation, I believe it is easier to understand what it is doing. In general, you include an interaction term because the product of two variables is important. The physical interpretation of that is different in each case.

Two dummy variables case: A dummy variable indicates something binary. In your example, the interaction term indicates whether someone is or isn't a black female. You could include this variable in a model without including the individual variables $black$ and $female$, and you could estimate the difference between black females and the rest of the population. If you have a model that already includes the variables $black$ and $female$, the interaction term might be redundant. After all, if we know that an individual being black increases life expectancy by one year, and being female increases life expectancy by two years, then we can estimate that a black female's life expectancy will be three years greater than the intercept. That is, an interaction term is redundant if the effects of the two individual variables are merely additive. You might think of this as the case where the two variables don't interact.

Often, that is not the case, and the two variables do somehow interact. Imagine that you are creating a model to predict the risk of imminent death based on current behavior. You might find that consuming alcohol increases the risk of imminent death by a moderate amount. Likewise, you may find that operating a motor vehicle increases the risk of imminent death by a moderate amount. But, you may also find that the case of consuming alcohol and operating a motor vehicle increases the risk of imminent death drastically! Much higher than the sum of the two individual effects. In that case, the interaction effect would model the difference between what you would expect the risk to be when you add the effects, and the actual risk when both effects are present. You might say it models the bonus risk. The case is the same in your example. The black*female term allows you to say that the life expectancy of a black female is the intercept +1 (because in general black individuals live longer) + 2 (because in general females live longer) and a bonus +0.5 (because black females live half a year longer than what would be predicted by the simple sum of the black and female effects).

One dummy variable and one continuous variable: In this case, it is probably easiest to interpret the interaction effect as the additional effect of each mg of calcium for individuals who are female. Consider the model $Height=5+.05*Female+0.1*Calcium+0.05Female*Calcium$. In this case, it is easy to see that for every mg of calcium, a male's height will increase by 0.1 ft, and a female's height will increase by 0.1 plus a bonus 0.05 ft.

Both variables are continuous case: In this case, you are modeling the product of two numeric features. In some cases, this might represent something specific and concrete. The interaction of height and weight can represent BMI, and the interaction of property length and property width represents the property area. In those cases, the coefficient of the term may be simple to understand (each additional square foot adds a bonus $10 to home value.)

In other cases, the interaction term might represent something completely abstract. In your example, you might say the interaction effect of a 6ft person with 100 IQ is "600 IQ points times feet". That is pretty meaningless on its own. However, you can still interpret the coefficient of that term in a general way. For example, if the coefficient of the IQ-Height interaction term is positive, you know that people who are tall and smart have an added bonus that isn't explained by the effect of being tall plus the effect of being smart.

Note that this interpretation is an oversimplification for simplicity. Adding an interaction term will change the estimates and meaning of the other coefficients. Consider case one. With no interaction term, the $female$ coefficient represents the difference in the life expectancy of females and non-females. However, with the interaction effect in the model, the effect of $female$ is now shared with the interaction term. The individual $female$ coefficient models the effect of being female only when the person is not black.