Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 2 down vote favorite

I have the following question, any hint would be really welcome:

I am trying to conduct a two-country comparison by running two separate regressions, one for each country, and testing $H_0:\,b_1=b_2$ (using Wald tests), where $b_1$ is the coefficient of an explanatory variable in regression group 1 and $b_2$ is the coefficient of the same explanatory variable in regression group 2 (the two groups have unequal size). However, I have came up with the following outcome: $b_1$ is statistically significant, $b_2$ is statistically insignicant, and $H_0$ cannot be rejected, i.e. the difference between the two coefficients is statistically insignificant.

I know that this outcome is possible in terms of statistics, but it sounds a bit "problematic" in terms of logic. How could someone sensibly justify it?

share|improve this question

1 Answer

up vote 6 down vote accepted

Conceptually, estimates are fuzzy. Think of them like this on an invisible horizontal number line:

Fuzzy numbers

(I don't show the number line itself because the actual values, as we will see, do not matter. You need only understand the metaphor that numerical difference = horizontal distance.)

$b_1$ is shown in cyan; it has a relatively small horizontal spread. The value it is estimating should be somewhere beneath the darker cyan points, possibly under the lighter cyan ones.

$b_2$ is shown in red; it has a relatively large horizontal spread. The value it is estimating should be somewhere beneath the darker red points, possibly under the lighter red ones.

For "significance," both are being compared to a definite number, shown by the horizontal location of the black line. (Usually this number is zero for slopes, but it need not be.)

$b_1$ is "significant" because, despite being fuzzy, it is clearly separated from the line.

$b_2$ is not "significant" because, although the value it estimates may differ from the line, it is so fuzzy that the separation is not clear.

$b_1$ is not significantly different from $b_2$ because they overlap substantially.

share|improve this answer
Oh, I see, many thanks for the reply; so, if I understand properly, given the fuzzy estimates, deriving conclusion about effect differences between groups is pointless. I hope though that fuzzy estimates do not imply that the model is poorly built...Thanks again for the explanation! – Bill718 Jul 26 '12 at 21:15
1  
It's not always pointless, Bill: it depends on the outcome. If, for instance, that cyan blob were far to the right of the red blob, we could conclude the two are significantly different. That's the whole point to testing: it helps us decide whether two estimates should be considered non-overlapping ("significant") or whether it's just too difficult to tell ("fail to reject the null hypothesis"). The amount of fuzziness, or horizontal spread, depends on the amount of independent data available and how much they vary: lots of fuzziness does not necessarily imply the model is poor. – whuber♦ Jul 26 '12 at 21:37
Sounds clear!thanks for your time and willingness to help! – Bill718 Jul 26 '12 at 21:48
And one more issue I was thinking before: I tried to pool the two samples together and detect effect differences by using interaction terms (between country dummy and the IV. Interaction term appears to be insignificant, and thus, effect difference is not found. I guess that the reason again is because the two estimates overlap substantially? – Bill718 Aug 5 '12 at 0:44

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.