I am using probability scoring on a data set where one variable has very small area of common support between treated and control (a small area in the middle exists). When I run a logit regression in R, I get the error message that "fitted probabilities numerically 0 or 1 occurred". A basic assumption in propensity scoring is that the probabilities lay between 0 and 1. What should be my next step to address a problem like this?

  • 2
    $\begingroup$ That variable almost completely predicts the allocation so why not just use it? $\endgroup$ – mdewey Apr 26 '17 at 16:03
  • 1
    $\begingroup$ This question seems clear enough to me. I'm voting to leave open. $\endgroup$ – gung - Reinstate Monica Apr 26 '17 at 16:27
  • 1
    $\begingroup$ @mdewey I agree it does predict allocation of the treatment very well but since there is so little common support I will not be able to balance the covariate meaning that any results from the final regression (of the outcome) will be biased? $\endgroup$ – robinsa Apr 26 '17 at 16:41
  • $\begingroup$ Leaving it also destroys the balance of all other covariates $\endgroup$ – robinsa Apr 26 '17 at 16:52

As you said, an assumption of causal inference is positivity, which is that all units have a nonzero probability of being in either treatment group. If this condition is violated, you essentially cannot make valid causal effect estimate without heroic assumptions. If you are willing to make an assumption, which is that the effect of that variable does not moderate treatment in any way, then after matching on the other variables, you can run a linear model including treatment and the covariate. The estimated coefficient for the treatment will be the causal effect of treatment assuming the effect of that covariate is the same across groups. This is essentially the logic of a regression discontinuity, where the treatment effect is only truly valid in the small area of common support but it is assumed that its effect would be constant across the range of the covariate.

  • $\begingroup$ I do not know what positivity means: do you mean that if X causes Y, then conditional on all possible confounders, the conditional mean of Y should show some difference if X takes two different values? Hence, if the OR for the propensity score is infinite, it's impossible to measure any such difference. $\endgroup$ – AdamO Mar 7 '18 at 15:33
  • $\begingroup$ Positivity means that all units have a nonzero probability of being in either treatment group. For units with a zero probability of being a treatment group, their counterfactual outcome for that treatment is not defined, and therefore the causal effect cannot be estimated for them. In practice, we can only estimate the probability of treatment with a model. If our model produces propensity scores of zero or one, it could indicate a lack of positive or an incorrect model. $\endgroup$ – Noah Mar 8 '18 at 15:57
  • $\begingroup$ The intuition behind my answer is that their model is wrong, that positivity is in effect, despite the estimated 0 or 1 probabilities. Therefore, they can combine propensity scores, a method that fails with this occurrence, with regression adjustment for the confounder, which is not affected by violations of positivity but requires extrapolation. $\endgroup$ – Noah Mar 8 '18 at 15:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.