Multiple regression with "overlapping" categorical dummy variables

Question

Using R, I want to run a linear regression to estimate the abnormal return on days with positive, negative and neutral news (CLASS). I'm a beginner in R, as well as in using regression models! First of all the data structure is as follows. CONTROLVAR just represents all the columns I use as control variables.

DATE <- c("1","2","3","4","5","6","7","1","2","3","4","5","6","7")
COMP <- c("A", "A", "A", "A", "A", "A", "A", "B", "B", "B", "B", "B", "B", "B")
RET <- c(-2.0,1.1,3,1.4,-0.2, 0.6, 0.1, -0.21, -1.2, 0.9, 0.3, -0.1,0.3,-0.12)
CLASS <- c("positive", "negative", "aneutral", "positive", "positive", "negative", "aneutral", "positive", "negative", "negative", "positive", "aneutral", "aneutral", "aneutral")
SUBJECT.1 <- c("LITIGATION","LAYOFF","POLLUTION","CHEMICAL DISASTER","PRESS RELEASE","PEOPLE","EMISSIONS","ENERGY","WASTE MANAGEMENT","EMPLOYEES","SUBJECT11","SUBJECT12","SUBJECT13","SUBJECT14")
SUBJECT.2 <- c("POLLUTION","EMPLOYEES","NUCLEAR","FUELS","STOCK OPTION PLAN","EXECUTIVES","CO2","SOLAR","POLLUTION","EXECUTIVES","SUBJECT21","SUBJECT22","SUBJECT23","SUBJECT24")
SUBJECT.3 <- c("ENVIRONMENT","JOB REDUCTIONS","POWER PLANTS","POLLUTION","EMPLOYEES","FRAUD","CLIMATE CHANGE","SUSTAINABILITY","HAZARDOUS WASTE","BONUS PAY","SUBJECT31","SUBJECT32","SUBJECT33","SUBJECT34")
CONTROLVAR <- c("11","13","13","14","13","14","12","11","13","13","14","13","14","12")

df <- data.frame(DATE, COMP, RET, CLASS, SUBJECT.1, SUBJECT.2, SUBJECT.3, CONTROLVAR, stringsAsFactors=F)

df

#    DATE COMP   RET    CLASS         SUBJECT.1         SUBJECT.2       SUBJECT.3 CONTROLVAR
# 1     1    A -2.00 positive        LITIGATION         POLLUTION     ENVIRONMENT         11
# 2     2    A  1.10 negative            LAYOFF         EMPLOYEES  JOB REDUCTIONS         13
# 3     3    A  3.00 aneutral         POLLUTION           NUCLEAR    POWER PLANTS         13
# 4     4    A  1.40 positive CHEMICAL DISASTER             FUELS       POLLUTION         14
# 5     5    A -0.20 positive     PRESS RELEASE STOCK OPTION PLAN       EMPLOYEES         13
# 6     6    A  0.60 negative            PEOPLE        EXECUTIVES           FRAUD         14
# 7     7    A  0.10 aneutral         EMISSIONS               CO2  CLIMATE CHANGE         12
# 8     1    B -0.21 positive            ENERGY             SOLAR  SUSTAINABILITY         11
# 9     2    B -1.20 negative  WASTE MANAGEMENT         POLLUTION HAZARDOUS WASTE         13
# 10    3    B  0.90 negative         EMPLOYEES        EXECUTIVES       BONUS PAY         13
# 11    4    B  0.30 positive         SUBJECT11         SUBJECT21       SUBJECT31         14
# 12    5    B -0.10 aneutral         SUBJECT12         SUBJECT22       SUBJECT32         13
# 13    6    B  0.30 aneutral         SUBJECT13         SUBJECT23       SUBJECT33         14
# 14    7    B -0.12 aneutral         SUBJECT14         SUBJECT24       SUBJECT34         12

This regression model would look like this:

mymodel1 <- lm(RET ~ CLASS + CONTROLVAR, data=df)

aneutral (neutral) will be the reference category. I would also like to see the effect of certain subjects of the article on the abnormal return. How can I do that? Let's say I want to include the subjects LITIGATION, POLLUTION and LAYOFF. I'd like to see how the effect of positive, negative and neutral news change, if the article is about POLLUTION for example. If I make "dummy columns" for the three subjects of interest, my data.frame looks like this:

DATE <- c("1","2","3","4","5","6","7","1","2","3","4","5","6","7")
COMP <- c("A", "A", "A", "A", "A", "A", "A", "B", "B", "B", "B", "B", "B", "B")
RET <- c(-2.0,1.1,3,1.4,-0.2, 0.6, 0.1, -0.21, -1.2, 0.9, 0.3, -0.1,0.3,-0.12)
CLASS <- c("positive", "negative", "aneutral", "positive", "positive", "negative", "aneutral", "positive", "negative", "negative", "positive", "aneutral", "aneutral", "aneutral")
SUBJECT.1 <- c("LITIGATION","LAYOFF","POLLUTION","CHEMICAL DISASTER","PRESS RELEASE","PEOPLE","EMISSIONS","ENERGY","WASTE MANAGEMENT","EMPLOYEES","SUBJECT11","SUBJECT12","SUBJECT13","SUBJECT14")
SUBJECT.2 <- c("POLLUTION","EMPLOYEES","NUCLEAR","FUELS","STOCK OPTION PLAN","EXECUTIVES","CO2","SOLAR","POLLUTION","EXECUTIVES","SUBJECT21","SUBJECT22","SUBJECT23","SUBJECT24")
SUBJECT.3 <- c("ENVIRONMENT","JOB REDUCTIONS","POWER PLANTS","POLLUTION","EMPLOYEES","FRAUD","CLIMATE CHANGE","SUSTAINABILITY","HAZARDOUS WASTE","BONUS PAY","SUBJECT31","SUBJECT32","SUBJECT33","SUBJECT34")
LITIGATION <- c(1,0,0,0,0,0,0,0,0,0,0,0,0,0)
POLLUTION  <- c(1,0,1,1,0,0,0,0,1,0,0,0,0,0)
LAYOFF     <- c(0,1,0,0,0,0,0,0,0,0,0,0,0,0)
CONTROLVAR <- c("11","13","13","14","13","14","12","11","13","13","14","13","14","12")

df2 <- data.frame(DATE, COMP, RET, CLASS, SUBJECT.1, SUBJECT.2, SUBJECT.3, LITIGATION, POLLUTION, LAYOFF, CONTROLVAR, stringsAsFactors=F)

df2 

#    DATE COMP   RET    CLASS         SUBJECT.1         SUBJECT.2       SUBJECT.3 LITIGATION POLLUTION LAYOFF CONTROLVAR
# 1     1    A -2.00 positive        LITIGATION         POLLUTION     ENVIRONMENT          1         1      0         11
# 2     2    A  1.10 negative            LAYOFF         EMPLOYEES  JOB REDUCTIONS          0         0      1         13
# 3     3    A  3.00 aneutral         POLLUTION           NUCLEAR    POWER PLANTS          0         1      0         13
# 4     4    A  1.40 positive CHEMICAL DISASTER             FUELS       POLLUTION          0         1      0         14
# 5     5    A -0.20 positive     PRESS RELEASE STOCK OPTION PLAN       EMPLOYEES          0         0      0         13
# 6     6    A  0.60 negative            PEOPLE        EXECUTIVES           FRAUD          0         0      0         14
# 7     7    A  0.10 aneutral         EMISSIONS               CO2  CLIMATE CHANGE          0         0      0         12
# 8     1    B -0.21 positive            ENERGY             SOLAR  SUSTAINABILITY          0         0      0         11
# 9     2    B -1.20 negative  WASTE MANAGEMENT         POLLUTION HAZARDOUS WASTE          0         1      0         13
# 10    3    B  0.90 negative         EMPLOYEES        EXECUTIVES       BONUS PAY          0         0      0         13
# 11    4    B  0.30 positive         SUBJECT11         SUBJECT21       SUBJECT31          0         0      0         14
# 12    5    B -0.10 aneutral         SUBJECT12         SUBJECT22       SUBJECT32          0         0      0         13
# 13    6    B  0.30 aneutral         SUBJECT13         SUBJECT23       SUBJECT33          0         0      0         14
# 14    7    B -0.12 aneutral         SUBJECT14         SUBJECT24       SUBJECT34          0         0      0         12

The first problem is, that the dummy variables partially overlap. My model would look something like this.

mymodel2 <- lm(RET ~ CLASS + LITIGATION + POLLUTION + LAYOFF  
               + LITIGATION * CLASS + POLLUTION * CLASS + LAYOFF * CLASS   # Interaction Variables
               + CONTROLVAR,     # Control Variables
               data=df2)

I'm quite sure this model is wrong, but I don't know exactly what is wrong and how to implement a model that works for this task. Can anyone help me with this problem? Thank You!

Answer 1

Why are you "quite sure" that it's wrong? I'm "quite sure" that it's right.

The "overlapping" case you're describing is called an "interaction." An interaction between two variables has the interpretation of creating a different slope for one variable at each level of the other variable. The most common case is between a continuous and categorical variable, so that the categorical variable can be said to "modify" the effect of the continuous one by changing the slope (do some algebra to convince yourself of this). But this interpretation extends to your case as well.

An interaction with a categorical variable is conceptually, but not statistically, equivalent to fitting a separate regression for each level of that categorical variable. The difference lies in the way the variance of the error is estimated, and therefore in the test statistics for significance.

You can also interpret an interaction between two categorical variables with $k_1$ and $k_2$ categories, respectively, as a single variable with $k_1\cdot k_2$ categories. Again, convince yourself of this.

However, you don't need to specify the main effect terms in your formula in R. The * operator inside a formula includes these implicitly. Instead you can use RET ~ CLASS * (LITIGATION + POLLUTION + LAYOFF) which expands to RET ~ CLASS * LITIGATION + CLASS * POLLUTION + CLASS * LAYOFF)

summary(mymodel2)
#
# Call:
# lm(formula = RET ~ CLASS * (LITIGATION + POLLUTION + LAYOFF), 
#     data = df2)
# 
# Residuals:
#      Min       1Q   Median       3Q      Max 
# -0.17333 -0.14875  0.00000  0.04125  0.33667 
# 
# Coefficients: (4 not defined because of singularities)
#                          Estimate Std. Error t value Pr(>|t|)    
# (Intercept)               0.04500    0.11750   0.383  0.71494    
# CLASSnegative             0.70500    0.20352   3.464  0.01340 *  
# CLASSpositive            -0.08167    0.17949  -0.455  0.66511    
# LITIGATION               -3.40000    0.33235 -10.230 5.09e-05 ***
# POLLUTION                 2.95500    0.26274  11.247 2.95e-05 ***
# LAYOFF                    0.35000    0.28782   1.216  0.26964    
# CLASSnegative:LITIGATION       NA         NA      NA       NA    
# CLASSpositive:LITIGATION       NA         NA      NA       NA    
# CLASSnegative:POLLUTION  -4.90500    0.38971 -12.586 1.54e-05 ***
# CLASSpositive:POLLUTION  -1.51833    0.37772  -4.020  0.00696 ** 
# CLASSnegative:LAYOFF           NA         NA      NA       NA    
# CLASSpositive:LAYOFF           NA         NA      NA       NA    
# ---
# Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# 
# Residual standard error: 0.235 on 6 degrees of freedom
# Multiple R-squared:  0.9816,    Adjusted R-squared:  0.9601 
# F-statistic: 45.73 on 7 and 6 DF,  p-value: 8.687e-05

One thing that is wrong: don't use df as a variable name in R, or for that matter data. One is the function for the F density, and the other one loads built-in data sets.