2
$\begingroup$

I am a bit stuck here. I am analyzing some data to see if there are heterogeneous treatment effects. To do that I am essentially running the three models below (and an Linear probability model (LPM)). My dependent variable is binary and I run logit models. Results Table

When I run two separate models (column 1 & 2) I find that for the group with the characteristic the treatment effect is significant not for the group without the characteristic. This leads me to think that there is a heterogeneous treatment effect. When I then run an overall model including the interaction term (column 3) this interaction is not significant.

I tried to understand what is going on here and searched different sources for explanations. Most sources pointed out that this is the case if there is collinearity issue with some other covariates in the model and that the characteristic needs to be interacted with all covariates in the full model. This cannot be a problem in my case as I have no other covariates in the model. My only independent variables are the characteristic and the treatment.

Can somebody explain to me when such a case can happen?

$\endgroup$
1
$\begingroup$

The effect of treatment for the characteristic group is different from 0, but the effect of treatment for the no charactertistci group is not different from 0, but it's also not different from the effect in the charactertistic group. This has to do with precision. Think about confidence intervals: the confidence interval for the effect of treatment in the no charactertistic group my exlcude 0, so we know the effect is different from 0. The confidence interval for the effect of treatment in the no characteristic group my include both 0 and the effect estimate in the characteristic group. In this way, it is distinguishable neither from 0 nor from the effect in the treatment group, and thus there is no interaction. Below is a schematic of what's happening:

          Treatment effect
--|-------------------------------
  0
 |----------^----------|       No C
      |----------^----------|  C

Also be careful how you interpret parameters for binary predictors in regression; -2.12 isn't exactly the effect of the interaction, it's the difference between treatment|characteristic and treatment|no_characteristic, which may not be where the real interaction is happening (i.e., the true difference might be between no_treatment|characteristic and no_treatment|no_characteristic).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.