2
$\begingroup$

In my study I have an independent continuous variable x1 (momentum) and four dummy variables D1 D2 D3 D4 which indicate industry type. I am investigating the four interaction variables between the dummy variable and momentum.

Question: do I have to test the interaction effects separately (i.e., one single model for each interaction, that is, one regression for x1D1, a different regression for x1D2...)?

Or do I need to test these three interactions effects in one single regression (i.e., in one single regression I include: x1D1 x1D2 x1D3…)? What is the difference in terms of interpretation?

Note: I have been investigating this for a while now and have come across two arguments:

In favor of a single regression: Include all the terms together so that you obtain the best possible estimates of your interaction terms. Fitting the models separately would mean failing to control for your other x-covariate.

In favor of multiple regressions: Including all the dummies and interactions can create multicollinearity problem. Therefore, it is suggested to include one dummy and interaction at a time.

Help would be appreciated, thank you in advance.

Spike Gontscharoff

$\endgroup$
  • $\begingroup$ Note that a single regression with all interactions is equivalent (in terms of the beta estimates) to fitting a separate model within each industry type. The standard errors are different (due to common/separate error variance parameters). $\endgroup$ – probabilityislogic May 19 '16 at 9:47
  • $\begingroup$ Hello, I tried estimating the model using sub samples but the results shows some sub samples being too small. Therefore I have switched to using dummy variables. Do you agree on estimating one model including all interaction variables compared to multiple regressions using a single interaction variable? $\endgroup$ – S. Gontscharoff May 19 '16 at 10:24
  • $\begingroup$ How big are the categories? You should be ok with 10 in each. Are you saying you've got less than 40 units in your samples? $\endgroup$ – probabilityislogic May 19 '16 at 13:09
  • $\begingroup$ The categorical variable (dummy, I was talking about) can take on 4 values: value 1 (2171 obs, 12 companies) , value 2 (3304 obs, 33 companies), value 3 (2492 obs, 21 companies), value 4 (5007 obs, 37 companies). I was doubting whether this is enough to perform your suggestion of sub samples. $\endgroup$ – S. Gontscharoff May 19 '16 at 14:05
  • $\begingroup$ @probabilityislogic do you think these sample sizes are sufficiently large enough in order to perform sub sample analysis? $\endgroup$ – S. Gontscharoff May 19 '16 at 19:14
2
$\begingroup$

Assuming the four dummies are not mutually exclusive categories$\dots$

You didn't give us an outcome variable, but let's assume this is continuous and call this $y$, you could specify the model: $$ y = b_0 + b_1 d_1 + b_2 d_2 + b_3 d_3 + b_4 d_4 + b_5 x_1 d_1 +b_6 x_1 d_2+b_7 x_1 d_3+b_8 x_1 d_4 + \epsilon $$ In R, much shorter to write:

mod<-lm(y~(d1+d2+d3+d4)*x1)

The marginal effect for a single dummy, let's say $d_1$, would then be: $$ \frac{\partial y}{\partial d_1} = b_1 + b_5x_1 $$

Unless you have a (a) high degree of multicollinearity, (b) a data set so small that with eight terms you have too few degrees of freedom, or (c) you have a theoretical reason for why you should also interact all your dummies with one another, something like the above should do the trick.

If for the single model, as written in your question, you interact everything-with-everything, that is a lot of terms, and your model will most likely be over-specified (unless you're working with a very large number of observations).

The argument against multiple different models would be that if two of your dummies are correlated with each other and the outcome variable, then leaving one out will produce omitted variable bias. But as ever, the 'best' model will depend on the best theoretical justification for the model and not just the best fit statistics.

If the dummy variables are mutually exclusive$\dots$

Then, both approaches are doing the same thing. You can estimate separate models for each one, or you can drop one of your dummy variables, in which case the dropped dummy variable will be your reference category. Both approaches should give you the same estimates, as noted by @probabilityislogic in the comments.

Update: Example

So, why will we get the same estimate, whether we interact or subset? First, let's generate some random data for our outcome $y$, assign each of our 20k observations to one of four mutually exclusive categories (e.g., industries), and generate another continuous covariate $x$.

set.sed(405)
d<-sample(1:4,20000,replace=TRUE)
x<-rnorm(20000,mean=5,sd=4)
y<-rnorm(20000,mean=10,sd=3)
data<-data.frame(cbind(y,d,x))
data$d<-factor(d)

Specify first model with interaction:

mod1<-lm(y~x*d,data=data)
summary(mod1)
Call:
lm(formula = y ~ x * d, data = data)

Residuals:
     Min       1Q   Median       3Q      Max 
-12.9035  -2.0169   0.0208   1.9962  13.0574 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept) 10.069148   0.068189 147.666   <2e-16 ***
x           -0.009853   0.010702  -0.921    0.357    
d2          -0.061505   0.096142  -0.640    0.522    
d3          -0.091567   0.096191  -0.952    0.341    
d4           0.078902   0.095918   0.823    0.411    
x:d2         0.010967   0.015106   0.726    0.468    
x:d3         0.016275   0.015162   1.073    0.283    
x:d4        -0.007537   0.015029  -0.501    0.616    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 2.997 on 19992 degrees of freedom
Multiple R-squared:  0.0002449, Adjusted R-squared:  -0.0001052 
F-statistic: 0.6995 on 7 and 19992 DF,  p-value: 0.6726

Now, calculate $\frac{\partial y}{\partial x}$ for when $d2 = 1$ and $d1 = d3 = d4 = 0$.

$$ -0.009853 + 0.010967*1 + 0.016275*0 + -0.007537*0 = 0.001114 $$

Now, let's subset the data to include only observations where $d2 = 1$, and regress $x$ on $y$.

mod2<-lm(y~x,data=subset(data,d=="2"))
summary(mod2)

Call:
lm(formula = y ~ x, data = subset(data, d == "2"))

Residuals:
     Min       1Q   Median       3Q      Max 
-10.5873  -2.0639   0.0417   2.0342  11.4410 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept) 10.007643   0.068723 145.623   <2e-16 ***
x            0.001114   0.010810   0.103    0.918    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 3.039 on 4982 degrees of freedom
Multiple R-squared:  2.13e-06,  Adjusted R-squared:  -0.0001986 
F-statistic: 0.01061 on 1 and 4982 DF,  p-value: 0.918

We get the exact same coefficient on $x$ of 0.001114. This is the effect of $x$ on $y$ when $d2=1$.

$\endgroup$
  • $\begingroup$ Hello, Thank you for your quick response. Some clarifications: the outcome variable is indeed continuous. My total sample size is around 20.000 observations. The dummy variables are categorical: each stock gets a code 1,2,3 or 4 (4 dummy variables) indicating the industry type. Observations are (roughly) evenly distributed for each category. I understand having an unbiased estimator is more important than having a ''nice'' answer. I will continue to run the model as a whole and look at the results. Again thank you! Note: I am using Stata, not R, but I now how to do this. $\endgroup$ – S. Gontscharoff May 19 '16 at 9:50
  • $\begingroup$ The dummy variables are mutually exclusive, in that each observation is assigned to only one of the four dummies? In that case, both approaches are doing exactly the same thing, as noted by @probabilityislogic. And you definitely don't want to include them all at once! Scan down to 'dummy variable trap' in this Wikipedia article. Most software will not allow you to do this, unless you force it to drop the intercept, which usually (not always, but usually) is a bad idea. $\endgroup$ – 5ayat May 19 '16 at 11:02
  • $\begingroup$ Thank you. Yes each observation is assigned to only one of the four dummies. I am aware that I include n-1 dummies to avoid perfect multicollinearity. The way I interpreted @probabilityislogic comment is to make sub samples instead of using dummy variables and performing a regression within each sub sample. In my case the dummy variables are indeed mutually exclusive. I am trying to understand your comment that in this case both approaches are doing the same thing. Thanks for these great answers! $\endgroup$ – S. Gontscharoff May 19 '16 at 11:44
  • $\begingroup$ 5ayat, you are a legend. It is clear to me now. I either do a regression with all dummy variables and all interaction variables. Or I estimate the model for each subsample. Your data sample finally helped me understand how these are related!. I was first doubting regarding the sample sizes but @probability made a very good comment about that. Thank you all! $\endgroup$ – S. Gontscharoff May 19 '16 at 14:17
  • $\begingroup$ definitely earned the check mark! One thing still remains unclear to me and perhaps I was not clear enough, my apologies. The categorical variables can take on a value from 1 to 10, however I am only interested in the values 1,2,3,4 (these are also the largest). Is is justified to either: 1) produce subsamples only for these four categories like you showed (data, d == "2"). 2) Remove the variables with number 5 till 10 and create dummy variables for 1,2,3 and 4? I am a bit confused now. Otherwise I would have to include 10 different dummy variables. Many thanks in advance!!! $\endgroup$ – S. Gontscharoff May 19 '16 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.