Skip to main content
Bounty Ended with 50 reputation awarded by ely
Edited to account for edits in the question.
Source Link
Bill
  • 8k
  • 32
  • 34

As a quick aside, usually you don't use a linear model to predict a binary treatment, so the model for predicting $T$ would typically be something like:

\begin{align} P(T=1) = \frac{exp(X\beta)}{1+exp(X\beta)} \end{align}

estimated by maximum likelihood. In the case of instrumental variables, this is a relatively unimportant detail, but in the case of propensity score matching, it is not a relatively unimportant detail---for PS matching, your model relating treatment to the predictors needs to be correct.

Usually when someone says that they are using propensity scores, what they mean is that they are using propensity score matching. The idea in propensity score matching is to take two groups of observations with the same propensity score but differing on treatment. I.e. the idea is to compare treatment vs control while controlling for the propensity score.

The idea of instrumental variables is kind of the opposite. What you are trying to do with instrumental variables is to ignore actual treatment and just look at how outcomes vary based on predicted treatment given the instrumental variables.

A simple/crude way to use propensity scores to estimate treatment effects would be to run the following regression by OLS:

\begin{align} Y = T^{\text{fit}} \gamma + T \delta + \epsilon \end{align}

Then you would call $\hat{\delta}_{\text{OLS}}$ your estimate of the treatment effect.

A similarly simple/crude use of instrumental variables would be to run the following regression by OLS:

\begin{align} Y = T^{\text{fit}} \alpha + \epsilon \end{align}

Then you would call $\hat{\alpha}_{\text{OLS}}$ your estimate of the treatment effect.

These are basically opposite strategies for estimating the effect of the treatment. The instrumental variables estimator looks only at variation in treatment induced by $X$: ie at $T^{\text{fit}}$. The PS matching estimator looks at variation in treatment, but removing variation in treatment induced by $X$.

Imagine parceling out the variation in treatment into two components: that explained by $X$ and that not explained by $X$. The IV approach considers the variation explained by $X$ to be "good" variation and variation not explained by $X$ as "bad" variation. The PS matching approach considers the variation explained by $X$ to be "bad" variation and variation not explained by $X$ as "good" variation.

If you ask, under what statistical assumptions is each of the estimators good, then you will see this distinction repeated. PS matching estimators are good estimators when treatment, after controlling for $X$, is exogenous. IV estimators are good estimators when $X$ is exogenous (has no direct effect on outcomes).

Whether each procedure works to reveal true treatment effects depends on what is true about the causal and statistical relationships in your application.

You might be interested in a paper I wrote with a colleague which is closely related to this point.

One last detail. The last regression equation in your question will not work---it will literally fail to estimate. As you have things set up, $T^{\text{fit}}$ is a linear function of $X$, so there will be perfect multicollinearity in the regression containing both as predictors.

As a quick aside, usually you don't use a linear model to predict a binary treatment, so the model for predicting $T$ would typically be something like:

\begin{align} P(T=1) = \frac{exp(X\beta)}{1+exp(X\beta)} \end{align}

estimated by maximum likelihood. In the case of instrumental variables, this is a relatively unimportant detail, but in the case of propensity score matching, it is not a relatively unimportant detail---for PS matching, your model relating treatment to the predictors needs to be correct.

Usually when someone says that they are using propensity scores, what they mean is that they are using propensity score matching. The idea in propensity score matching is to take two groups of observations with the same propensity score but differing on treatment. I.e. the idea is to compare treatment vs control while controlling for the propensity score.

The idea of instrumental variables is kind of the opposite. What you are trying to do with instrumental variables is to ignore actual treatment and just look at how outcomes vary based on predicted treatment given the instrumental variables.

A simple/crude way to use propensity scores to estimate treatment effects would be to run the following regression by OLS:

\begin{align} Y = T^{\text{fit}} \gamma + T \delta + \epsilon \end{align}

Then you would call $\hat{\delta}_{\text{OLS}}$ your estimate of the treatment effect.

A similarly simple/crude use of instrumental variables would be to run the following regression by OLS:

\begin{align} Y = T^{\text{fit}} \alpha + \epsilon \end{align}

Then you would call $\hat{\alpha}_{\text{OLS}}$ your estimate of the treatment effect.

These are basically opposite strategies for estimating the effect of the treatment. The instrumental variables estimator looks only at variation in treatment induced by $X$: ie at $T^{\text{fit}}$. The PS matching estimator looks at variation in treatment, but removing variation in treatment induced by $X$.

Imagine parceling out the variation in treatment into two components: that explained by $X$ and that not explained by $X$. The IV approach considers the variation explained by $X$ to be "good" variation and variation not explained by $X$ as "bad" variation. The PS matching approach considers the variation explained by $X$ to be "bad" variation and variation not explained by $X$ as "good" variation.

If you ask, under what statistical assumptions is each of the estimators good, then you will see this distinction repeated. PS matching estimators are good estimators when treatment, after controlling for $X$, is exogenous. IV estimators are good estimators when $X$ is exogenous (has no direct effect on outcomes).

Whether each procedure works to reveal true treatment effects depends on what is true about the causal and statistical relationships in your application.

You might be interested in a paper I wrote with a colleague which is closely related to this point.

One last detail. The last regression equation in your question will not work---it will literally fail to estimate. As you have things set up, $T^{\text{fit}}$ is a linear function of $X$, so there will be perfect multicollinearity in the regression containing both as predictors.

Usually when someone says that they are using propensity scores, what they mean is that they are using propensity score matching. The idea in propensity score matching is to take two groups of observations with the same propensity score but differing on treatment. I.e. the idea is to compare treatment vs control while controlling for the propensity score.

The idea of instrumental variables is kind of the opposite. What you are trying to do with instrumental variables is to ignore actual treatment and just look at how outcomes vary based on predicted treatment given the instrumental variables.

A simple/crude way to use propensity scores to estimate treatment effects would be to run the following regression by OLS:

\begin{align} Y = T^{\text{fit}} \gamma + T \delta + \epsilon \end{align}

Then you would call $\hat{\delta}_{\text{OLS}}$ your estimate of the treatment effect.

A similarly simple/crude use of instrumental variables would be to run the following regression by OLS:

\begin{align} Y = T^{\text{fit}} \alpha + \epsilon \end{align}

Then you would call $\hat{\alpha}_{\text{OLS}}$ your estimate of the treatment effect.

These are basically opposite strategies for estimating the effect of the treatment. The instrumental variables estimator looks only at variation in treatment induced by $X$: ie at $T^{\text{fit}}$. The PS matching estimator looks at variation in treatment, but removing variation in treatment induced by $X$.

Imagine parceling out the variation in treatment into two components: that explained by $X$ and that not explained by $X$. The IV approach considers the variation explained by $X$ to be "good" variation and variation not explained by $X$ as "bad" variation. The PS matching approach considers the variation explained by $X$ to be "bad" variation and variation not explained by $X$ as "good" variation.

If you ask, under what statistical assumptions is each of the estimators good, then you will see this distinction repeated. PS matching estimators are good estimators when treatment, after controlling for $X$, is exogenous. IV estimators are good estimators when $X$ is exogenous (has no direct effect on outcomes).

Whether each procedure works to reveal true treatment effects depends on what is true about the causal and statistical relationships in your application.

You might be interested in a paper I wrote with a colleague which is closely related to this point.

Source Link
Bill
  • 8k
  • 32
  • 34

As a quick aside, usually you don't use a linear model to predict a binary treatment, so the model for predicting $T$ would typically be something like:

\begin{align} P(T=1) = \frac{exp(X\beta)}{1+exp(X\beta)} \end{align}

estimated by maximum likelihood. In the case of instrumental variables, this is a relatively unimportant detail, but in the case of propensity score matching, it is not a relatively unimportant detail---for PS matching, your model relating treatment to the predictors needs to be correct.

Usually when someone says that they are using propensity scores, what they mean is that they are using propensity score matching. The idea in propensity score matching is to take two groups of observations with the same propensity score but differing on treatment. I.e. the idea is to compare treatment vs control while controlling for the propensity score.

The idea of instrumental variables is kind of the opposite. What you are trying to do with instrumental variables is to ignore actual treatment and just look at how outcomes vary based on predicted treatment given the instrumental variables.

A simple/crude way to use propensity scores to estimate treatment effects would be to run the following regression by OLS:

\begin{align} Y = T^{\text{fit}} \gamma + T \delta + \epsilon \end{align}

Then you would call $\hat{\delta}_{\text{OLS}}$ your estimate of the treatment effect.

A similarly simple/crude use of instrumental variables would be to run the following regression by OLS:

\begin{align} Y = T^{\text{fit}} \alpha + \epsilon \end{align}

Then you would call $\hat{\alpha}_{\text{OLS}}$ your estimate of the treatment effect.

These are basically opposite strategies for estimating the effect of the treatment. The instrumental variables estimator looks only at variation in treatment induced by $X$: ie at $T^{\text{fit}}$. The PS matching estimator looks at variation in treatment, but removing variation in treatment induced by $X$.

Imagine parceling out the variation in treatment into two components: that explained by $X$ and that not explained by $X$. The IV approach considers the variation explained by $X$ to be "good" variation and variation not explained by $X$ as "bad" variation. The PS matching approach considers the variation explained by $X$ to be "bad" variation and variation not explained by $X$ as "good" variation.

If you ask, under what statistical assumptions is each of the estimators good, then you will see this distinction repeated. PS matching estimators are good estimators when treatment, after controlling for $X$, is exogenous. IV estimators are good estimators when $X$ is exogenous (has no direct effect on outcomes).

Whether each procedure works to reveal true treatment effects depends on what is true about the causal and statistical relationships in your application.

You might be interested in a paper I wrote with a colleague which is closely related to this point.

One last detail. The last regression equation in your question will not work---it will literally fail to estimate. As you have things set up, $T^{\text{fit}}$ is a linear function of $X$, so there will be perfect multicollinearity in the regression containing both as predictors.