I have a set of 460 survey responses of students evaluating their teachers' teaching style. Students are from different schools within several countries. I have separately identified that student evaluation scores differ significantly by both country and institution.
The question is, then, whether region (such as Scandinavia, German-speaking Europe, etc.) or institution, with each variable coded through dummies, has the more relevant effect on evaluation scores. The set has 16 schools grouped into 3 regions. The results are additionally controlled by other dummies such as gender, age group and degree pursued. The regression includes interaction effects for all IVs.
Below are the standardized coefficients of only the significant IVs (IV combinations with high multicolinearity were automatically excluded by the software):
Source Value Std.err t Pr > |t|
Region B 0,234 0,046 5,089 < 0,0001
NotMotherTongue*School B 0,209 0,051 4,055 < 0,0001
Age22-28*School R 0,163 0,042 3,912 0,000
School L 0,146 0,042 3,471 0,001
School B 0,120 0,052 2,328 0,020
MA degree*School W 0,098 0,042 2,346 0,019
Region C*Age35+ 0,094 0,042 2,264 0,024
Female -0,102 0,042 -2,423 0,016
Female*School A -0,127 0,042 -3,062 0,002
School O -0,158 0,045 -3,499 0,001
The problem here is that I followed the advice that one should code dummies by leaving out the category with most cases. Most responses were in region A, hence I only dummy coded the two other regions (B, C). However, since most schools are located in region A, I am also interested in the interaction effect between region A and the schools located in that region.
I therefore run the same regression but with Region A as dummy, and Regions B+C ( totaling 82 responses) as reference category. Standardized coefficients of only the significant IVs are here:
Source Value Standard error t Pr > |t|
NotMotherTongue*School B 0,209 0,051 4,055 < 0,0001
Age22-28*School R 0,166 0,042 3,988 < 0,0001
School B 0,123 0,052 2,374 0,018
MA degree*Schoo 0,101 0,042 2,430 0,015
Age35+*MA degree 0,089 0,044 2,028 0,043
Female -0,103 0,043 -2,415 0,016
Female*School A -0,125 0,042 -3,016 0,003
School T -0,137 0,048 -2,827 0,005
School O -0,156 0,044 -3,560 0,000
Region A -0,302 0,050 -6,059 < 0,0001
This second result set shows many similar results, yet also differences, and especially the effect of Region A becomes obvious.
Question: which of these two regression scenarios is the more useful one for discerning whether the impact of school or of the region is greater on the evaluation scores? To me, it seems that both regression results should be interpreted together. Is this legitimate?
Question: which IV has the greater impact, school or region? To me, it appears that for both regression results, the region has the largest standardized coefficient. But each result set also features 7 schools with significant influence. Overall, I am not sure how to interpret the results.
EDIT: below are the model parameters (not standardized coefficients) including intercept, with dummy coding regions B/C and region A as reference. I removed the categories degree and age22-28 to reduce complexity.
Source Value Standard error
Intercept 2,350 0,103 22,796 < 0,0001
NotMotherTongue*School B 0,958 0,334 2,869 0,004
NotMotherTongue*Age29-34 0,695 0,260 2,679 0,008
School R 0,371 0,144 2,576 0,010
School A 0,596 0,240 2,487 0,013
School B 0,483 0,210 2,303 0,022
NotMotherTongue*School R -0,899 0,404 -2,227 0,027
Female*School H -0,581 0,276 -2,106 0,036
Female*School L 0,797 0,385 2,069 0,039
School C 0,562 0,279 2,016 0,044
*(below not significant)*
Female*Region C -0,619 0,332 -1,867 0,063
NotMotherTongue*School L 0,605 0,376 1,611 0,108
NotMotherTongue*School F -0,355 0,248 -1,431 0,153
NotMotherTongue*School K -0,558 0,398 -1,401 0,162
Female*School R -0,248 0,182 -1,364 0,173
Age29-34*School L -0,713 0,556 -1,283 0,200
School F 0,230 0,181 1,275 0,203
School K 0,224 0,181 1,239 0,216
NotMotherTongue*Region C -0,404 0,336 -1,203 0,230
Female*School B -0,364 0,318 -1,144 0,253
Female*Age35+ 0,197 0,175 1,126 0,261
Age35+*School R 0,410 0,370 1,107 0,269
Female*School W 0,185 0,171 1,084 0,279
Region C*Age35+ 0,394 0,365 1,080 0,281
Age35+*School K -0,345 0,322 -1,071 0,285
Female -0,124 0,119 -1,049 0,295
Female*School I 0,183 0,179 1,019 0,309
Region C 0,303 0,322 0,942 0,347
NotMotherTongue*Region B -0,531 0,574 -0,924 0,356
NotMotherTongue*School R -0,359 0,404 -0,888 0,375
Region B -0,207 0,239 -0,868 0,386
NotMotherTongue*Age35+ 0,219 0,267 0,822 0,412
Female*School R 0,190 0,236 0,808 0,419
NotMotherTongue*School I -0,339 0,423 -0,801 0,423
Age35+*School T 0,227 0,290 0,782 0,435
Female*NotMotherTongue 0,142 0,185 0,770 0,442
School R -0,166 0,215 -0,770 0,442
Region C*Age29-34 -0,432 0,562 -0,768 0,443
Age35+*School L -0,329 0,440 -0,746 0,456
NotMotherTongue*School C 0,425 0,609 0,697 0,486
School G 0,126 0,193 0,653 0,514
Female*School K -0,144 0,231 -0,624 0,533
Age35+ -0,156 0,253 -0,614 0,540
Female*School A -0,262 0,433 -0,606 0,545
Age29-34*School W 0,278 0,481 0,578 0,564
Female*School F -0,092 0,207 -0,445 0,656
Age35+*School I -0,126 0,291 -0,433 0,666
School H 0,077 0,204 0,376 0,707
Age29-34*School R -0,208 0,568 -0,367 0,714
Age29-34 -0,160 0,443 -0,360 0,719
Age29-34*School F -0,181 0,519 -0,349 0,727
Age29-34*School T 0,191 0,550 0,347 0,729
Female*Region B 0,124 0,392 0,318 0,751
School T -0,043 0,155 -0,279 0,780
School I 0,038 0,142 0,266 0,790
NotMotherTongue 0,055 0,207 0,266 0,790
School L -0,062 0,342 -0,182 0,856
Age35+*School C 0,112 0,625 0,180 0,857
Age29-34*School K -0,085 0,508 -0,167 0,867
NotMotherTongue*School A 0,118 0,714 0,165 0,869
Female*School T -0,029 0,184 -0,159 0,873
Age29-34*School R -0,077 0,514 -0,149 0,882
School W -0,017 0,135 -0,124 0,901
NotMotherTongue*School G -0,055 0,505 -0,109 0,913
Age35+*School R 0,039 0,505 0,076 0,939
Region B*Age35+ -0,034 0,582 -0,059 0,953
Female*School G -0,010 0,248 -0,039 0,969
Age29-34*School I -0,015 0,475 -0,032 0,974
Age35+*School G 0,013 0,526 0,024 0,981
Female*School C -0,009 0,493 -0,017 0,986
Age29-34*School H 0,003 0,538 0,005 0,996
Female*Age29-34 0,001 0,174 0,005 0,996
NotMotherTongue*School T 0,001 0,342 0,004 0,997
Region B*Age29-34 0,001 0,543 0,003 0,998
The issue is: as the effect of region A is visible from the intercept, how do I interpret the intercept coefficient, given that a) the intercept is not shown in the standardized coefficients, making direct effect comparison of the coefficients difficult, and b) the intercept reflects not only the reference region (A) but also the reference school, as well as all other reference categories?
Also, in all of these models I have let the software compute all possible interactions (hence a long list of model coefficients). Is that acceptable?