Regression with/without interaction vis a vis CEF

Question

I am interested in improving my understanding of regression and CEF. In particular, I bring two related questions: 1) How to interpret a relationship with two dummy variables without interaction; and 2) What is stata doing when the interaction code ## is used. I've been trying to replicate in different packages and have no idea what is happening there.

I've created the following data to use as an example.

female  white score
0   0   4.9
1   0   6.4
0   0   5.8
1   1   9.5
0   1   7.1
0   1   7.5
1   1   8.4
0   1   6.2
0   0   5.1
1   1   9.1

From here, we have the following conditional means:

$E[score|white=0]=5.5$

$E[score|white=1]=7.96$

$E[score|female=0]=6.1$

$E[score|female=1]=8.35$

That can also be obtained by simple linear regression,

$score = \beta_0 + \beta_1 \times Female$ or $score = \gamma_0 + \gamma_1 \times White$

Then, $\beta_0=6.1=E[score|female=0]$ and $\beta_1=2.25=E[score|female=1]-E[score|female=0]$. That is, the simple regression gives difference between conditional means. Similar results happen for a reationship with the race dummy.

Now, the CEF for two dummies is

$E[score|female,white] = \beta_0 + \beta_1 \times Female + \beta_2 \times White + \beta_3 \times Female \times White$

And we get that

$\beta_0=5.26=E[score|white=0,female=0]$

$\beta_1=1.13=E[score|white=0,female=1]-\beta_0$

$\beta_2=1.66=E[score|white=1,female=0]-\beta_0$

$\beta_2=3.73=E[score|white=1,female=1]-\beta_0$

Up to this point, everything corresponds to textbooks and previous posts. But what happens when we run a multiple linear regression without an interaction?

$score = \beta_0 + \beta_1 Female + \beta_2 White$

Where $\beta_0 = 5.111$, $\beta_1=1.75$ and $\beta_2 =1.97$ do not represent conditional means. What are they?

My additional question corresponds to Stata. Here it is said that # is different from ## and indeed stata gives different results. When using # Stata will show me differences in conditional means as in the interaction above. When using ##, Stata will show similar results but the coefficient for $female=1,white=1$ is $.933$. All of this contradicts the answer from Statalist, as # gave me the CEF/Interactions model but ## gives something else.

Answer 1

There are two related ways to think about your first problem:

1. you need a fully saturated model for the CEF to match the summary statistics
2. omitted variable bias will mess up your CEF

I will start with the second. When you omit the interaction, you get omitted variable bias. Instead of the "true" model

$$score= \beta_0 + \beta_1 \times Female + \beta_2 \times White + \beta_3 \times Female \times White + \varepsilon$$ you are using $$score= \tilde \beta_0 + \tilde \beta_1 \times Female + \tilde \beta_2 \times White + \tilde \epsilon$$

Since the omitted variable is (1) positively correlated with female and white, and (2) has a non-zero association with score, its omission is contaminating the other coefficients, which then alters the predictions. By leaving it out, gender and race can only have additive effects, with no special change from female and white together. This will bias the female and white coefficients up (since they now also include the scores of folks who are both). It will also bias the mean down. This means the coefficients/predictions will not have a clean relationship to the 4 original means.

Here's an example showing how to fix the bias for the intercept:

. clear

. input female  white score

        female      white      score
  1. 0   0   4.9
  2. 1   0   6.4
  3. 0   0   5.8
  4. 1   1   9.5
  5. 0   1   7.1
  6. 0   1   7.5
  7. 1   1   8.4
  8. 0   1   6.2
  9. 0   0   5.1
 10. 1   1   9.1
 11. end

. /* Summary Stats */
. table female white, c(mean score)

------------------------------
          |       white       
   female |        0         1
----------+-------------------
        0 | 5.266667  6.933333
        1 |      6.4         9
------------------------------
 
. /* Fully Saturated Model: Should Match SS above */
. reg score i.female##i.white

      Source |       SS           df       MS      Number of obs   =        10
-------------+----------------------------------   F(3, 6)         =     21.90
       Model |  21.3866659         3  7.12888864   Prob > F        =    0.0012
    Residual |  1.95333428         6  .325555713   R-squared       =    0.9163
-------------+----------------------------------   Adj R-squared   =    0.8745
       Total |  23.3400002         9  2.59333336   Root MSE        =    .57057

------------------------------------------------------------------------------
       score |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
    1.female |   1.133333   .6588431     1.72   0.136    -.4787977    2.745464
     1.white |   1.666667   .4658725     3.58   0.012     .5267177    2.806615
             |
female#white |
        1 1  |   .9333334   .8069148     1.16   0.291    -1.041116    2.907783
             |
       _cons |   5.266667   .3294216    15.99   0.000     4.460601    6.072732
------------------------------------------------------------------------------

. margins female#white

Adjusted predictions                            Number of obs     =         10
Model VCE    : OLS

Expression   : Linear prediction, predict()

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
female#white |
        0 0  |   5.266667   .3294216    15.99   0.000     4.460601    6.072732
        0 1  |   6.933333   .3294216    21.05   0.000     6.127268    7.739399
        1 0  |        6.4   .5705749    11.22   0.000     5.003854    7.796147
        1 1  |          9   .3294216    27.32   0.000     8.193934    9.806066
------------------------------------------------------------------------------

. /* A Non-Saturated Model (Dropping the Interaction Term) */
. reg score i.female i.white

      Source |       SS           df       MS      Number of obs   =        10
-------------+----------------------------------   F(2, 7)         =     30.70
       Model |  20.9511103         2  10.4755552   Prob > F        =    0.0003
    Residual |   2.3888899         7  .341269985   R-squared       =    0.8976
-------------+----------------------------------   Adj R-squared   =    0.8684
       Total |  23.3400002         9  2.59333336   Root MSE        =    .58418

------------------------------------------------------------------------------
       score |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
    1.female |   1.755556   .3894555     4.51   0.003     .8346398    2.676471
     1.white |   1.977778   .3894555     5.08   0.001     1.056862    2.898693
       _cons |   5.111111   .3078916    16.60   0.000     4.383063    5.839159
------------------------------------------------------------------------------

. margins female#white

Adjusted predictions                            Number of obs     =         10
Model VCE    : OLS

Expression   : Linear prediction, predict()

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
female#white |
        0 0  |   5.111111   .3078916    16.60   0.000     4.383063    5.839159
        0 1  |   7.088889   .3078916    23.02   0.000     6.360841    7.816937
        1 0  |   6.866667   .4130799    16.62   0.000     5.889888    7.843446
        1 1  |   8.844444   .3078916    28.73   0.000     8.116397    9.572492
------------------------------------------------------------------------------

. /* Correct the wrong non-saturated intercept for omitted interaction term bias */
. gen white_x_female = white*female

. regress white_x_female i.female i.white

      Source |       SS           df       MS      Number of obs   =        10
-------------+----------------------------------   F(2, 7)         =     11.20
       Model |         1.6         2          .8   Prob > F        =    0.0066
    Residual |          .5         7  .071428571   R-squared       =    0.7619
-------------+----------------------------------   Adj R-squared   =    0.6939
       Total |         2.1         9  .233333333   Root MSE        =    .26726

------------------------------------------------------------------------------
white_x_fe~e |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
    1.female |   .6666667   .1781742     3.74   0.007     .2453517    1.087982
     1.white |   .3333333   .1781742     1.87   0.104    -.0879816    .7546483
       _cons |  -.1666667    .140859    -1.18   0.275    -.4997454     .166412
------------------------------------------------------------------------------

. display 5.111111 - (.9333334 )*(_b[_cons] + _b[1.female]*0 + _b[1.white]*0)
5.2666666

Note that this recovers the original intercept from the saturated model. Obviously this is not feasible in general.

Now on to your second question. When you tell Stata to include x##z, it will include an x, a z, and an x*z term. This is binary operator to specify factorial interactions. It is pronounced "factorial cross", or "factorial octothorpe" if you want people to look at you funny.

When you tell Stata to use x#z, it will put in indicators for each combination of levels of x and z: a set of two-way or pairwise interactions. Its name is cross.

In other words, x##z is the same as x z x#z.

However, there are some complications. With continuous variables, c.x#c.z on its own means x*z as a regressor, plus a constant. With categorical variables, something counterintuitive happens. Two binary variables, specified as i.x#i.z on their own, will get you three pairwise interactions and a constant. These will produce the same predictions as the saturated model, but some of the coefficients will not be the same because the model is parameterized differently. If you add, the , coefl option you can see what Stata is doing under the hood.

People often expect the lone i.x#i.z model to have a constant and an x*z term only (for x=1,z=1), but that is just not how it works.

For completeness, a lone i.x#c.z will get you two-pairwise interactions, plus a constant.