Multiple regression - interpreting interaction effect with dummy IVs

Question

I have a set of 460 survey responses of students evaluating their teachers' teaching style. Students are from different schools within several countries. I have separately identified that student evaluation scores differ significantly by both country and institution.

The question is, then, whether region (such as Scandinavia, German-speaking Europe, etc.) or institution, with each variable coded through dummies, has the more relevant effect on evaluation scores. The set has 16 schools grouped into 3 regions. The results are additionally controlled by other dummies such as gender, age group and degree pursued. The regression includes interaction effects for all IVs.

Below are the standardized coefficients of only the significant IVs (IV combinations with high multicolinearity were automatically excluded by the software):

Source                      Value   Std.err   t     Pr > |t|    

Region B                    0,234   0,046   5,089   < 0,0001    
NotMotherTongue*School B    0,209   0,051   4,055   < 0,0001    
Age22-28*School R           0,163   0,042   3,912   0,000   
School L                    0,146   0,042   3,471   0,001   
School B                    0,120   0,052   2,328   0,020   
MA degree*School W          0,098   0,042   2,346   0,019   
Region C*Age35+             0,094   0,042   2,264   0,024   
Female                     -0,102   0,042   -2,423  0,016   
Female*School A            -0,127   0,042   -3,062  0,002   
School O                   -0,158   0,045   -3,499  0,001   

The problem here is that I followed the advice that one should code dummies by leaving out the category with most cases. Most responses were in region A, hence I only dummy coded the two other regions (B, C). However, since most schools are located in region A, I am also interested in the interaction effect between region A and the schools located in that region.

I therefore run the same regression but with Region A as dummy, and Regions B+C ( totaling 82 responses) as reference category. Standardized coefficients of only the significant IVs are here:

Source                      Value   Standard error  t   Pr > |t|

NotMotherTongue*School B    0,209   0,051   4,055   < 0,0001
Age22-28*School R           0,166   0,042   3,988   < 0,0001
School B                    0,123   0,052   2,374   0,018
MA degree*Schoo             0,101   0,042   2,430   0,015
Age35+*MA degree            0,089   0,044   2,028   0,043
Female                      -0,103  0,043   -2,415  0,016
Female*School A             -0,125  0,042   -3,016  0,003
School T                    -0,137  0,048   -2,827  0,005
School O                    -0,156  0,044   -3,560  0,000
Region A                    -0,302  0,050   -6,059  < 0,0001

This second result set shows many similar results, yet also differences, and especially the effect of Region A becomes obvious.

1. Question: which of these two regression scenarios is the more useful one for discerning whether the impact of school or of the region is greater on the evaluation scores? To me, it seems that both regression results should be interpreted together. Is this legitimate?

2. Question: which IV has the greater impact, school or region? To me, it appears that for both regression results, the region has the largest standardized coefficient. But each result set also features 7 schools with significant influence. Overall, I am not sure how to interpret the results.

EDIT: below are the model parameters (not standardized coefficients) including intercept, with dummy coding regions B/C and region A as reference. I removed the categories degree and age22-28 to reduce complexity.

Source                      Value   Standard error  

Intercept                   2,350   0,103   22,796  < 0,0001
NotMotherTongue*School B    0,958   0,334   2,869   0,004
NotMotherTongue*Age29-34    0,695   0,260   2,679   0,008
School R                    0,371   0,144   2,576   0,010
School A                    0,596   0,240   2,487   0,013
School B                    0,483   0,210   2,303   0,022
NotMotherTongue*School R    -0,899  0,404   -2,227  0,027
Female*School H             -0,581  0,276   -2,106  0,036
Female*School L             0,797   0,385   2,069   0,039
School C                    0,562   0,279   2,016   0,044
*(below not significant)*
Female*Region C             -0,619  0,332   -1,867  0,063
NotMotherTongue*School L    0,605   0,376   1,611   0,108
NotMotherTongue*School F    -0,355  0,248   -1,431  0,153
NotMotherTongue*School K    -0,558  0,398   -1,401  0,162
Female*School R             -0,248  0,182   -1,364  0,173
Age29-34*School L           -0,713  0,556   -1,283  0,200
School F                    0,230   0,181   1,275   0,203
School K                    0,224   0,181   1,239   0,216
NotMotherTongue*Region C    -0,404  0,336   -1,203  0,230
Female*School B             -0,364  0,318   -1,144  0,253
Female*Age35+               0,197   0,175   1,126   0,261
Age35+*School R             0,410   0,370   1,107   0,269
Female*School W             0,185   0,171   1,084   0,279
Region C*Age35+             0,394   0,365   1,080   0,281
Age35+*School K             -0,345  0,322   -1,071  0,285
Female                      -0,124  0,119   -1,049  0,295
Female*School I             0,183   0,179   1,019   0,309
Region C                    0,303   0,322   0,942   0,347
NotMotherTongue*Region B    -0,531  0,574   -0,924  0,356
NotMotherTongue*School R    -0,359  0,404   -0,888  0,375
Region B                    -0,207  0,239   -0,868  0,386
NotMotherTongue*Age35+      0,219   0,267   0,822   0,412
Female*School R             0,190   0,236   0,808   0,419
NotMotherTongue*School I    -0,339  0,423   -0,801  0,423
Age35+*School T             0,227   0,290   0,782   0,435
Female*NotMotherTongue      0,142   0,185   0,770   0,442
School R                    -0,166  0,215   -0,770  0,442
Region C*Age29-34           -0,432  0,562   -0,768  0,443
Age35+*School L             -0,329  0,440   -0,746  0,456
NotMotherTongue*School C    0,425   0,609   0,697   0,486
School G                    0,126   0,193   0,653   0,514
Female*School K             -0,144  0,231   -0,624  0,533
Age35+                      -0,156  0,253   -0,614  0,540
Female*School A             -0,262  0,433   -0,606  0,545
Age29-34*School W           0,278   0,481   0,578   0,564
Female*School F             -0,092  0,207   -0,445  0,656
Age35+*School I             -0,126  0,291   -0,433  0,666
School H                    0,077   0,204   0,376   0,707
Age29-34*School R           -0,208  0,568   -0,367  0,714
Age29-34                    -0,160  0,443   -0,360  0,719
Age29-34*School F           -0,181  0,519   -0,349  0,727
Age29-34*School T           0,191   0,550   0,347   0,729
Female*Region B             0,124   0,392   0,318   0,751
School T                    -0,043  0,155   -0,279  0,780
School I                    0,038   0,142   0,266   0,790
NotMotherTongue             0,055   0,207   0,266   0,790
School L                    -0,062  0,342   -0,182  0,856
Age35+*School C             0,112   0,625   0,180   0,857
Age29-34*School K           -0,085  0,508   -0,167  0,867
NotMotherTongue*School A    0,118   0,714   0,165   0,869
Female*School T             -0,029  0,184   -0,159  0,873
Age29-34*School R           -0,077  0,514   -0,149  0,882
School W                    -0,017  0,135   -0,124  0,901
NotMotherTongue*School G    -0,055  0,505   -0,109  0,913
Age35+*School R             0,039   0,505   0,076   0,939
Region B*Age35+             -0,034  0,582   -0,059  0,953
Female*School G             -0,010  0,248   -0,039  0,969
Age29-34*School I           -0,015  0,475   -0,032  0,974
Age35+*School G             0,013   0,526   0,024   0,981
Female*School C             -0,009  0,493   -0,017  0,986
Age29-34*School H           0,003   0,538   0,005   0,996
Female*Age29-34             0,001   0,174   0,005   0,996
NotMotherTongue*School T    0,001   0,342   0,004   0,997
Region B*Age29-34           0,001   0,543   0,003   0,998

The issue is: as the effect of region A is visible from the intercept, how do I interpret the intercept coefficient, given that a) the intercept is not shown in the standardized coefficients, making direct effect comparison of the coefficients difficult, and b) the intercept reflects not only the reference region (A) but also the reference school, as well as all other reference categories?

Also, in all of these models I have let the software compute all possible interactions (hence a long list of model coefficients). Is that acceptable?

Answer 1

First, you don't need to code the way you did in your second regression to find out the interaction effects of schools in region a. You should look at ALL the coefficients and also REPORT them, no matter if they are significant or not.

The intercept in your first regression will show you the effect of region A and the reference school (I cannot tell which one it is) on the teacher's teaching style. Of course, if you use more IVs, then this value will also be connected to the reference values of all other IVs. Then, estiamtion school B will give you the estimation of teaching style for school B in region A with all other IVs on their reference values etc. The interaction region B: school B - implies the value of a teaching style of school B in region B as compared to the reference school in region A etc.

So, for the questions

1) Both regressions are good, but it's better when you don't code B and C together but separate - then you don't loose valuable information

2) Here again you have to give us all coefficients. And I believe that your model has too many IVs with too many levels. You should try to make it simpler in order to be able to interpret the data. But just at looking at the regression 1, we see that changing from region A to region B has greater impact then changing from the reference school to school L...

Summa summarum - too many IVs with too many levels, you have to keep it simpler to be able to interpret the data!

EDIT (answer to additional questions):

1) how do I interpret the intercept coefficient, given that a) the intercept is not shown in the standardized coefficients, making direct effect comparison of the coefficients difficult

First, to make clear - none of these coefficients is standardized. So to interpret the intercept, let's first look at a simple regression formula $y=intercept + ax$. You interpret the intercept as the value in criterion (in your case the evaluation on the teacher's style - you know more about the scale) when the values of all other predictors are 0.

Now, it is important that 0 means something sensible for you for each predictor, so that you might be able to interpret the data. For example, if you have a variable age (on an intervale scale), intercept would pinpoint to a value somebody with 0 years would have, which is of course not useful. In order to make it more useful, people usually centralize the variables around mean value. Centralizing means substracting the mean from each score in the variable. So the SD stays the same, but now the mean would be 0. Then you can interpret intercept (in the age example) as the value of teacher's evaluation for people with mean age in your sample.

P.S. I am not sure what you concretely meant with "direct comparison of effects" - which effects precisely?

2) how do I interpret the intercept coefficient, given that the intercept reflects not only the reference region (A) but also the reference school, as well as all other reference categories?

Well yeah, that's the problem when you have multiple predictors. You cannot interpret the intercept only for region A, without taking account of reference schools, or other predictors. But that's the point of putting in more predictors. If you want a mean effect of region A, independent on the school, just leave the school or any other predictor out of the equation.

Also, in all of these models I have let the software compute all possible interactions (hence a long list of model coefficients). Is that acceptable?

Well, depending how you look at it. Practically - yes, if the model converged, and it obviously did (sometimes it's problem to calculate so many parameters if you have few participants - the standard errors go very high). On the other hand, theoretically - it is questionable - would your theory predict those interactions? There you will find an answer...