Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 4 down vote favorite
2

So I run logistic regressions of same dependent variables and independent variables on two sets of data $S_1$ and $S_2$. I obtained two sets of coefficients $\beta_1$ and $\beta_2$. Now I suspect that $\beta_1$ and $\beta_2$ should really be equal, since I don't think there are distinctive difference between $S_1$ and $S_2$.

My question is, how do I test the significant differences between the coefficients?

Thanks!

share|improve this question
One thing to keep in mind: In logistic regression, the coefficients and variance are not separately identified. As a result, comparing estimates from separate models run on separate data is misleading. Greg Snow's answer below avoids this problem since everything is estimated in the same model. – Charlie Dec 30 '11 at 1:24
But since in the two models we generally impose that the variance is the same, why can't we compare the results of the two models? – Manoel Galdino Dec 30 '11 at 12:20

1 Answer

up vote 6 down vote accepted

You can stack the 2 datasets on top of each other with an indicator variable for which dataset the data comes from. Example (simplified):

> d1
  y x
1 0 1
2 0 2
3 1 3
> d2
  y x
1 0 2
2 1 3
3 1 4
> d3
  y x g
1 0 1 0
2 0 2 0
3 1 3 0
4 0 2 1
5 1 3 1
6 1 4 1

Then fit a model with interactions with the grouping variable, i.e. 

logit(y) ~ b0 + b1*x + b2*g + b3*x*g

The test on b2 will measure the difference in the intercepts (compare to your seperate fits) and b3 will measure the difference in the slopes (compare again), so tests on b2 and b3 are the tests of interest. If you have additional covariates then just include them as well with the interaction terms.

share|improve this answer
+1 for this answer. I would just like to add a nuance. Of course, b2 and b3 are the coefficients of interest here. However, you have to test if they are simultaneously equal to zero, and not separately. – user5644 Dec 28 '11 at 21:04
Hi Greg. A follow-up question. How do I test if b2 and b3 are equal to zero? I know in linear regression there is t-stat, but not sure if there is such an equivalence in logistic regression. One way I know regarding feature selection is I can test if the deviance changed much after removing b2*g and b3*x*g from the logistic regression, but is there any more straightforward metrics like t-stat in logistic regression? Thanks! – CodeNoob Dec 29 '11 at 14:05
3  
You can do a Wald-Z test on a specific coefficient in a logistic regression, but this tests each coefficient individually and if you really want to see if there is a difference between groups then you need to test them together (as @lejohn pointed out). Also the z-test can suffer from the Hauk-Donner effect sometimes where a very large coefficient is not seen as significant because the variance is overestimated. Better is to use a full/reduced model liklihood ratio test. – Greg Snow Dec 29 '11 at 16:22
Great answer! One question, wouldn't this approach suffer if one of the data sets were much smaller than the other? Say you had 10 rows in d1 and 100 rows in d2. If so, is there a way of compensate for the different sizes of the data sets? – Rasmus Bååth Mar 21 at 15:15
@RasmusBååth, you get better power when the sample sizes are equal, but the validity and general properties of the test should not suffer from unequal sample sizes. If there is a specific concern then you can simulate data and test out the proceedure to see how it behaves. – Greg Snow Mar 21 at 18:02

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.