Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 2 down vote favorite

I am a bit puzzled with the following issue: the outcome of one-way ANOVA test shows that the mean difference of variable y between two country samples is statistically significant. However, after pooling the two samples together and running an OLS regression with y as dependent variable, some other IV and by including a country dummy, the effect of the country dummy appears to be statistically insignificant, implying that there is no country-related effect on dependent variable y. Any explanation for this outcome?

share|improve this question
Was the other IV included in the ANOVA? – gung Aug 3 '12 at 18:53

3 Answers

up vote 9 down vote

If I'm right in guessing that the other IV was not included in the ANOVA, then the most likely reason is that the two countries differed on the other IV, and that the countries only look 'significant' if the other IV is omitted. I wonder if the other IV is 'significant'? With sufficient confounding, it may not be. One last (but I think unlikely) possibility is that when you added the IV you lost 1 degree of freedom, and so if the IV were totally unrelated to the response, you would have lost a trivial amount of statistical power.

share|improve this answer
Thanks a lot for your reply!Actually yes, I did not include the other IVs (sorry I was no clear before, I have various IVs;I have also created some interaction terms between country dummy and some of the IVs to detect country-related differences for specific predictors) in the ANOVA test actually. Some of the IVs are significant in the regression analysis. However, after some experiments with the IVs in the model, I noticed that in some cases the country dummy may remain significant, but with the opposite sign... – Bill718 Aug 3 '12 at 19:28
up vote 5 down vote

In addition to the other answers, you wrote

the effect of the country dummy appears to be statistically insignificant, implying that there is no country-related effect on dependent variable y.

That is not what a non-significant result implies. In fact, unless the means on Y for the two countries were identical (extremely unlikely), you know that there was a difference in your sample. And the chance that there was a 0 difference in the population is infinitesimal.

A non-significant result does not imply anything. It simply says that, if the difference in the population were 0, you could get a test statistic this high more than 5% of the time.

share|improve this answer
Sound clear!!appreciated! – Bill718 Aug 4 '12 at 1:02
up vote 2 down vote

This may be the same as gung's answer but it looks at it slightly differently and I think this could help understanding and interpretation. The IV is highly correlated with country. Therefore if the country indicator helps predict the response the IV could too. So by themselves each would be significant but included together only one will. It would therefore probably be best for prediction especially if the sample size is small to include just one of the two. Which one to choose would only matter if you care about causation. The problem this causes is often called confounding which is part of what gung was discussing.

share|improve this answer
Thanks!!!So, confouding, if I understand properly, causes the so-called endogeneity issue... In this example, I found out that the correlation between one specific IV and the country dummy is rather high indeed. That IV refers to weather conditions in different regions of each country, so it sounds reasonable to absorb some of the explanatory power of the dummy! – Bill718 Aug 3 '12 at 20:27

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.