1
$\begingroup$

In a multiple regression, the regression coefficient of the two subsamples is opposite to that of the full sample. Both coefficients are significant. The sum of the number of obs in the two subsamples is equal to that of the full sample. Is this normal? What are the possible causes? Thanks.

$\endgroup$
2
  • 6
    $\begingroup$ Sounds like a case of Simpson's paradox (- see the diagram in particular). What are these subsamples? Are you splitting on the values of one dichotomous variable? $\endgroup$
    – Glen_b
    Aug 14, 2013 at 21:31
  • $\begingroup$ Yes, I split the data based on whether they are pre-year 2000 or post-year 2000 $\endgroup$
    – Tammy
    Aug 14, 2013 at 22:04

1 Answer 1

2
$\begingroup$

Consider the following data (low noise, so you can see it with small sample size)

 x    y   g
 1  16.1  1
 2  16.9  1
 3  17.9  1
 4  19.1  1
10   5.9  2
11   7.1  2
12   8.1  2
13   8.9  2

If $g$ represented the before vs the after, then each could have a line with slope coefficient of one sign, while the overall data (ignoring the variable $g$) had a line with slope coefficient of the opposite sign:

simpson's paradox

The red and black lines are individual regression lines, the blue line the overall line. The right thing to do in this data is to add the factor $g$ as a predictor in the regression model (e.g. to add an indicator variable that is "1" when in the second group and "0" otherwise), either as main effect only (for parallel lines) or with interaction (for two lines of different slope).

This is an example of Simpson's paradox - though it's not particularly paradoxical, since it's really just failing to account for an important variable.

So yes, it's possible for example, for the level to change between the two periods, which could result in flipping the sign when you ignore the time variable, as in the diagram above. Since it's quite easy for this to happen, my guess would be Simpson's paradox.

$\endgroup$
5
  • 3
    $\begingroup$ An example from Peter Huber, amazon.com/Data-Analysis-Learned-Probability-Statistics/dp/… y is expenditure on cosmetics, x is income, g is gender. In each case expenditure increases with income, but women have higher intercept and lower average income. Similar to the diagram above: schematically women are black data points, men are red. I'm not clear whether the data were real. $\endgroup$
    – Nick Cox
    Aug 15, 2013 at 18:28
  • $\begingroup$ That's very interesting! I just tried including a dummy for the time variable. The full sample results are now consistent with those of the two subsamples!!! Now it's time to go back and figure out why this has happened. Thanks a bunch for enlightening me on this issue. $\endgroup$
    – Tammy
    Aug 15, 2013 at 18:52
  • 1
    $\begingroup$ Simpson's paradox is a veridical paradox in Quine's sense. What seems false or impossible is seen to be true or possible on more careful examination. See en.wikipedia.org/wiki/… $\endgroup$
    – Nick Cox
    Aug 16, 2013 at 1:00
  • 1
    $\begingroup$ @NickCox Yes, I get that it formally counts as a paradox in that sense. I guess my issue is that many of those (and that is one, at least for me) seem rather less paradoxical than others. But it's an issue I've been aware of so long (and that I have explained so many times) that I guess I've lost the sense that it can be so surprising. $\endgroup$
    – Glen_b
    Aug 16, 2013 at 1:05
  • 1
    $\begingroup$ I agree. The difficulties in statistics for me are two-fold (1) recognising it in a fresh dataset (even though you have met it before), especially when it is only one facet of a pattern of relations (2) needing a graph to make it clear, to oneself and to others. Do people who work with tables all the time find it easy to spot in tables, even with all categorical data? In regression problems, you really need a graph. $\endgroup$
    – Nick Cox
    Aug 16, 2013 at 1:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.