I am estimating the relationship between students' math scores and using a supplemental math software. I've adjusted for some demographic variables in the model. In my first model, I use the full dataset including students from grade 3, 4, and 5. In my model, I adjust for the grade coefficient I get a significant estimate of 5.1 (units) for the treatment group (those that used the program). I, then, stratify the dataset into 3rd graders only, 4th graders only, and 5th graders only and I fit a separate model per each {same model as the one fitted to the full model but this time without grade covariate}. The estimated treatment effect are: 1) Grade 3: 7.4 units 2) Grade 4: 12.29 units (significant) 3) Grade 5: 12.81 units (marginally significant)
As you may see, the overall effect of 5.1 is not between the estimates of the stratified datasets. Why does this happen? Is this strange to you?
