I'm having a question regarding, how should the sign of returned coefficients (e.g. by R's lm()) in linear model be interpreted.

Particularly I was doing a model on some kids' test scores and there's a categorical variable motiv $\in \{0,1\}$ where $1$ signifies whether the kid was motivated to study and $0$ signifies whether the kid was not motivated.

I'm using the following model to lm():

fit2 <- lm(dta$X.U.FEFF..mpist. ~ dta$matem + dta$aidink + dta$motiv)

where X.U.FEFF..mpist. is the test score, matem is the kid's mathematics score, aidink is the kid's score in native language and then there's the motiv variable.

lm() returns the following model:

$$mpist = 266.296 + 23.226 matem + 13.725 aidink + (-5.416) motiv$$

one notices that the coefficient for motiv is negative. But this is illogical considering that $motiv == 0$ corresponds to no motivation and $motiv == 1$ corresponds to motivation. So by this, if the kid has been motivated, this would result in a slightly lower test score, but if the kid has not been motivated, this would result in a slightly higher test score.

So how should one interpret a situation like this? Is the sign of the coefficient generated for motiv reasonable?

  • $\begingroup$ You don't mention whether the coefficient is significant. If it's not statistically significantly different from zero then it's not necessarily strange that the sign is the unexpected direction. $\endgroup$ – Ian_Fin Oct 5 '16 at 14:47
  • $\begingroup$ @Ian_Fin So using R's summary() on my model gives dta$motiv a PR(>|t|) value of 0.203, which is > 0.05 which means that the variable is not significant? $\endgroup$ – mavavilj Oct 5 '16 at 14:48
  • $\begingroup$ Exactly! Or more accurately, it's not significant at the $\alpha=.05$ level. If the true value of the coefficient (i.e. the effect that exists in the world) is 0, then the coefficient that you estimate will be negative in approximately 50% of your samples. $\endgroup$ – Ian_Fin Oct 5 '16 at 14:54

Do not forget that the model shows the effect of each variable given the others in the model. It is perfectly possible that motivation does affect test score but you cannot see that because you have included two other variables, maths score and language score, which may be affected by motivation so the effect of motivation on test after allowing for maths score and language score may be small. To diagnose further why not see what the mean test score is for each level of motivation?

  • $\begingroup$ So you're saying that using only motivation might show a positive correlation, but adding variables such as maths and native language variables seems to display that motivation might actually have a negative correlation (in light of the variables math and native language)? $\endgroup$ – mavavilj Oct 5 '16 at 16:06
  • $\begingroup$ In fact as others have pointed out it has no real effect but yes, what you suggest may happen. $\endgroup$ – mdewey Oct 5 '16 at 16:18
  • $\begingroup$ Yeah, it's just that I have not got a proper intuition on when to trust the "significance test" for variables. $\endgroup$ – mavavilj Oct 5 '16 at 16:55
  • 1
    $\begingroup$ So I tried taking the means: mean(dta[dta$motiv == 0,]$X.U.FEFF..mpist.) is 526.9299 and mean(dta[dta$motiv == 1,]$X.U.FEFF..mpist.) is 553.8646. So these are means from the original data, not from predictions using the model. But what I'm supposed to see from these? This seems to display positive correlation of motivation and test score? $\endgroup$ – mavavilj Oct 5 '16 at 17:04

The estimated coefficient is reasonable in sense that it is a value optimal from the model's perspective given your data. You mentioned in comment that it is not statistically significant, what means that it has large errors. The value of this parameter does not seem to have much practical significance since it decreases the initial score of 266.296 (intercept) by just five points -- I guess that this is a negligible effect on the final score?

As about the negative sign and the fact that it is not significant, my first guess would be that simply your motiv variable is a poor one. Motivation is not something of a binary nature, it's simply not that you are motivated, or you aren't, there is a huge gray area in between. How was it measured? Was it a questionary item that kids were asked? Maybe it was a question to their teachers, or parents? If so, maybe it is just teachers who can't judge if kids are motivated, or not...

  • $\begingroup$ So you seem to think that it's possible to have "negative correlation" in motiv and test score? Eventhough this sounds counterintuitive. $\endgroup$ – mavavilj Oct 5 '16 at 15:25
  • $\begingroup$ @mavavilj First it is not significant, it has large errors. It is like you had a clock that gives you time approximately to $\pm$12 hours -- it is simply unreliable. $\endgroup$ – Tim Oct 5 '16 at 17:22
  • $\begingroup$ But only with the mathematics and native language variables? It seems to have correct correlation on its own. $\endgroup$ – mavavilj Oct 5 '16 at 17:24
  • $\begingroup$ But maybe the question is also, so it's unreliable in relation to actual test results? I.e. it doesn't work very well in predicting them? So in that sense the motiv does not work in this particular model very well. $\endgroup$ – mavavilj Oct 5 '16 at 17:25
  • $\begingroup$ @mavavilj yes, exactly as you wrote. When controlling for the other variables it does not make a difference. $\endgroup$ – Tim Oct 5 '16 at 17:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.