4
$\begingroup$

In causal inference, the consistency assumption states that there are no multiple versions of treatment. Specifically, for a potential outcome unit $Y_i$ and a binary treatment vector $\mathbf{Z}$,

$$ Y_i(\mathbf{Z})=Y_i(\mathbf{Z'}) \ \ \forall \ \mathbf{Z},\mathbf{Z'}:\mathbf{Z}=\mathbf{Z'} $$ In literature, it says that "This says that the mechanism used to assign the treatments does not matter and assigning the treatments in a different way does not constitute a different treatment."

I am wondering how to make sense of this equation. What is it actually trying to say?

$\endgroup$
3
  • 1
    $\begingroup$ Provide a specific reference. In statistics, consistency means probability limits. When people assign a different meaning to a commonly used term, this is the surest way to sow confusion. $\endgroup$ – StasK Sep 25 '17 at 4:27
  • $\begingroup$ I am referring to pg. 5, assumption 1 in this paper: arxiv.org/pdf/1609.06245.pdf $\endgroup$ – user321627 Sep 25 '17 at 5:04
  • 1
    $\begingroup$ I think this is related to the distinction I tried to clarify here. $\endgroup$ – Dimitriy V. Masterov Sep 25 '17 at 5:31
7
$\begingroup$

Let me use $X$ for the treatment, $Y$ for the observed outcome and $Y(x)$ for the potential outcome under $X = x$.

Consistency means that for an individual $i$, his observed outcome $Y_i$ when $X_i = x$ is his potential outcome $Y_{i}(x)$. Or, more formally:

$$X_i = x \implies Y_i(x) = Y_i$$

When the treatment is binary ($X \in \{0,1\}$) consistency translates to the well known equation:

$$ Y_i = X_i Y_i(1) +(1-X_i)Y_i(0) $$

In an informal way, that's what people intend to convey when stating that the "way" $X$ is assigned doesn't matter, that is, that there aren't multiple versions of the treatment. For if there were multiple potential outcomes to the same $x$, when you see $X_i = x$, then which potential outcome is the observed outcome $Y_i$? And if it doesn't matter how the treatment is assigned, then when $X_i = x$, $Y_i(x)$ is well defined and usually assumed to be equal to $Y_i$ (but note that, even without multiple versions of the treatment, you still need to assume that $X_i = x \implies Y_i(x) = Y_i$).

What would happen without consistency?

Consistency is what connects the potential outcomes with the observed data. That is, it's consistency that allows us writing things like:

$$ E[Y(x)|X = x] = E[Y|X =x] $$

Which transforms expressions of counterfactual quantities $Y(x)$ into expressions of observed quantities $Y$.

Without consistency, all of your potential outcomes data would be "missing". To make this clear, consider again a binary treatment. If consistency holds , when $X=1$ you observe $Y_i = Y(1)$ and when $X = 0$ you observe $Y_i = Y_i(0)$. The other two potential outcomes are unobserved, so your potential outcomes table would look like this:

\begin{array} {|r|r|r|r|} \hline &Y_i(1) & Y_i(0) \\ \hline X_i=1 &Y_i &unobserved\\ \hline X_i=0 & unobserved & Y_i\\ \hline \end{array}

If consistency did not hold, this is what you would get --- you don't observe any potential outcome:

\begin{array} {|r|r|r|r|} \hline &Y_i(1) & Y_i(0) \\ \hline X_i=1 &unobserved &unobserved\\ \hline X_i=0 & unobserved & unobserved\\ \hline \end{array}

Consistency, potential outcomes,axioms of counterfactuals and structural causal models

In some of the potential outcomes literature, consistency is not explicitly defined, and casually considered together with other substantive assumptions such as SUTVA. In the axiomatization of counterfactuals, consistency is seen as a corollary of the axiom of composition. However, once you properly define a structural causal model (SCM) and you define counterfactuals as derived from interventional submodels, consistency is simply a natural consequence that automatically holds for all SCMs (see Gales and Pearl and also Pearl's Causality, Chapter 7).

Finally, whether consistency really "holds" in your model when compared to the real world is a practical/modeling issue. That is, to make any inference, you always need consistency, so what you are question is not the rule, but modeling assumptions. For example, do you think you have properly defined the treatment assignment $X$? Or are there more relevant things in the assignment of $X$ that would matter for the outcome that you didn't model? These are the questions you need to think about to judge whether your model is a good approximation for the problem you are investigating.

$\endgroup$
4
  • 1
    $\begingroup$ When you say that there is no multiple versions of the treatment, is there an easy to explain example of this? Is it just saying that the mapping from treatment to potential outcomes is a bijection? $\endgroup$ – user321627 Jan 24 '18 at 0:17
  • $\begingroup$ @user321627 no, it just needs to be well defined. $Y(x) = x^2$ is not a bijection for example and it is a valid potential outcome. $\endgroup$ – Carlos Cinelli Jan 24 '18 at 0:58
  • 1
    $\begingroup$ Is an example of breaking consistency if say there was an experiment to test efficacy of aspirin on headaches. The treatment was an aspirin dose and control was not taking aspirin, and the treatment dosage was not standard in that some subjects got 500 mg while others got 30 mg? $\endgroup$ – user321627 Jan 24 '18 at 1:18
  • 2
    $\begingroup$ @user321627 yes, if X = 1 for taking aspirin and X = 0 for not taking aspiring but X = 1 corresponds to 500mg and 30mg and Y(500mg) != Y(30mg), then the same X =1 can have different potential outcomes. $\endgroup$ – Carlos Cinelli Jan 24 '18 at 1:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.