5
$\begingroup$

I am performing linear regression on a dataset and want to include an interaction between sex and another covariate. However, in my model sex is coded as $0$ for female and $1$ for male.

If I have an interaction of the form sex*covariate then my design matrix may look something like this

$$\begin{bmatrix} 0 & 2.3 & 0 \\ 1 & 3.3 & 3.3 \\ 1 & 9.0 & 9.0 \\ 0 & 7.3 & 0 \\ 0 & 5.2 & 0 \\ 1 & 2.2 & 2.2 \\\end{bmatrix}$$

Where the first column is sex the second is my covariate and the third is the interaction between sex and the covariate.

This seems really strange to me as essentially it doesn't matter what the values in the second column were for women, the interaction will always be zero. Suppose instead I had coded sex in my design as -1 and 1 for female and male. Then I would have:

$$\begin{bmatrix} -1 & 2.3 & -2.3 \\ 1 & 3.3 & 3.3 \\ 1 & 9.0 & 9.0 \\ -1 & 7.3 & -7.3 \\ -1 & 5.2 & -5.2 \\ 1 & 2.2 & 2.2 \\\end{bmatrix}$$

My question is this; doesn't it matter how I code sex? Which of the two above designs are correct for including an interaction here? And why?


Edit: My original post did not mention but my design also includes an intercept. So in fact the above matrices should look like:

$$\begin{bmatrix} 1 & 0 & 2.3 & 0 \\ 1 & 1 & 3.3 & 3.3 \\ 1 & 1 & 9.0 & 9.0 \\ 1 & 0 & 7.3 & 0 \\ 1 & 0 & 5.2 & 0 \\ 1 &1 & 2.2 & 2.2 \\\end{bmatrix}$$

and:

$$\begin{bmatrix} 1 &-1 & 2.3 & -2.3 \\ 1 &1 & 3.3 & 3.3 \\ 1 &1 & 9.0 & 9.0 \\ 1 &-1 & 7.3 & -7.3 \\ 1 & -1 & 5.2 & -5.2 \\ 1 & 1 & 2.2 & 2.2 \\\end{bmatrix}$$

respectively.

$\endgroup$
  • 8
    $\begingroup$ I love the title. It would attract a great amount of interest on other sites, I imagine ;-). $\endgroup$ – whuber May 28 at 18:54
  • 3
    $\begingroup$ Purely research I promise ;) $\endgroup$ – JDoe2 May 28 at 20:37
5
$\begingroup$

Algebra lights the way.

The purpose of an "interaction" between a binary variable like gender and another variable (let's just call it "$X$") is to model the possibility that how a response (call it "$Y$") is associated with $X$ may depend on the binary variable. Specifically, it allows for the slope (aka coefficient) of $X$ to vary with gender.

The desired model, without reference to how the binary variable might be encoded, therefore is

$$\eqalign{ E[Y\mid \text{Male}, X] &= \phi(\alpha + \beta_{\text{Male}} X) \\ E[Y\mid \text{Female}, X] &= \phi(\alpha + \beta_{\text{Female}} X). }\tag{*}$$

for some function $\phi.$

One way--by far the commonest--to express this model with a single formula is to create a variable "$Z$" that indicates the gender: either $Z=1$ for males and $Z=0$ for females (the indicator function of $\text{Male}$ in the set $\{\text{Male},\text{Female}\}$) or the other way around with $Z=1$ for females and $Z=0$ for males (the indicator function of $\text{Female}$). But there are other ways, of which the most general is to

encode males as some number $Z=m$ and some different number $Z=f$ for females.

(Because $m\ne f,$ division by $m-f$ below is allowable.)

However we encode the binary variable, we may now express the model in a single formula as

$$E[Y\mid X] = \phi(\alpha + \beta Y + \gamma Z X)$$

because, setting

$$\gamma = \frac{\beta_{\text{Male}} - \beta_{\text{Female}}}{m - f}\tag{**}$$

and

$$\beta = \beta_{\text{Male}} - \gamma m = \beta_{\text{Female}} - \gamma f,$$

for males with $Z=m$ this gives

$$\phi(\alpha + \beta X + \gamma Z X) = \phi(\alpha + (\beta + \gamma m)X) = \phi(\alpha + \beta_{\text{Male}})X$$

and for females with $Z=f,$

$$\phi(\alpha + \beta X + \gamma Z X) = \phi(\alpha + (\beta + \gamma fX) = \phi(\alpha + \beta_{\text{Female}})X$$

which is exactly model $(*).$


The expression for $\gamma$ in $(**)$ is crucial: it shows how to interpret the model.

For instance, when using the indicator for males, $m-f = 1-0$ and $\gamma$ is the difference between the male and female slopes in the model. When using the indicator for females, $m-f = 0-1 = -1$ and now $\gamma$ is the difference computed in the other direction: between the female and male slopes.

In the example of the question where $m=1$ and $f=-1,$ now

$$\gamma = \frac{\beta_{\text{Male}} - \beta_{\text{Female}}}{m - f} = \frac{\beta_{\text{Male}} - \beta_{\text{Female}}}{2} \tag{**}$$

is half the difference in slopes.

Despite these differences in interpretation of the coefficient $\gamma,$ these are all equivalent models because they are all identical to $(*).$

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Ah! Great! Thank you! This explanation makes very clear intuitive sense! Thanks again @whuber $\endgroup$ – JDoe2 May 29 at 13:29
2
$\begingroup$

If you have an interaction with sex, then this means that you create a new variable that did not exist before.

For instance:

  • let the outcome (dependent variable) be the probability of a baby
  • let sex be a variable which is either 0 or 1
  • and let's say we interact it with condom use which is either 0 or 1 as well.

Then you could have some table like the following (I make these numbers up as an example but try to approach realistic values):

Probability of having a baby

                      Yes Sex         No Sex
Unprotected           0.50             0
Condom                0.01             0

So this could be modelled with two fixed effects like

$$\text{$y = a + b$ sex $+c$ unprotected}$$

But you won't get it right. The above formula will give

                       Yes Sex         No Sex
Unprotected            a+b+c           a+c
Condom                 a+b             a

This has only three variables to determine 4 values. If you try to make unprotected sex equal to 0.5 by giving some weight to b or c then you get that protected sex or no sex will have too much weight.

When you add an interaction term then you get

$$\text{$y = a + b$ sex $+c$ unprotected $+ d$ sex and unprotected}$$

                       Yes Sex         No Sex
Unprotected            a+b+c+d         a+c
Condom                 a+b             a

So that is how your interaction with sex is helping to get babies.


You can give indeed different values to sex, this will change the weights. Also when you change the interaction term and where you intercept, then things get mixed up. It can change how significant the value of the intercept is, and depending on your interaction, the value of the fixed model effects change as well.

But for the total model prediction, the prediction for the probability of whether you get a baby, it does not matter. The values of the sexes and the interaction, their significance, should not be measured. An analysis of variance is better.

So when you got that fixed, then the point of the intercept becomes just a matter of convenience. I like to do like you and put it in between men and women by giving men and women equal, but opposite, weight -1 and +1. In that case, the factors will show the differences relative to a place that is in between men and women.


Quickie:

The model is equivalent in the prediction of the means as long as the column space remains the same (this is the case in your example when you include an intercept term), but particular statistical tests for coefficients may change.

See also

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Interesting answer, but I think you are interpreting sex as the sex act (yes/no), rather than the sex of a person (male/female). Your points are valid, but it does not really match the inquiry by the OP. $\endgroup$ – Ben May 28 at 23:17
  • $\begingroup$ Thank you for the detailed explanation! I did mean gender but the explanation is still invaluable! This has really helped me gain some intuition for when to use interactions! Thanks again! $\endgroup$ – JDoe2 May 29 at 13:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.