2
$\begingroup$

Context

Let us say we are interested in the Average Treatment Effect (ATE) as an estimand. Following the potential outcomes framework, we define it as: $$\frac{1}{N} \sum_{1}^{N}(Y_i^0 - Y_i^1)$$

where $Y_i^a$ is the potential outcome for subject $i$ under condition $A = a$, $A$ is a treatment assignment variable that takes values $0$ or $1$ for control and treatment, respectively, and $N$ is the number of subjects in the population.

Let us define two working models for the causal structure:

  • DAG 1, with a fork on $X1$ and on $X2$:

    • $A \leftarrow X1 \rightarrow Y$,
    • $A \leftarrow X2 \rightarrow Y$ and
    • $A \rightarrow Y$
  • DAG 2, with mediator $X1$ and a fork on $X2$:

    • $A \rightarrow X1 \rightarrow Y$,
    • $A \leftarrow X2 \rightarrow Y$ and
    • $A \rightarrow Y$

Clearly, in DAG 1 both $X1$ and $X2$ are confounders, whereas in DAG 2 only $X2$ confounds the relationship between $A$ and $Y$. Using a regression model as estimation strategy (OLS), we arrive to two models that depend on our causal assumptions (adding a subscript to indicate each one):

  • $Y_i = \beta_{10} + \beta_{11} A_i + \beta_{12} X1_i + \beta_{13} X2_i + \epsilon_i$, where we use our estimate of $\hat{\beta}_{11}$ as an estimate for the ATE.
  • $Y_i = \beta_{20} + \beta_{21} A_i + \beta_{22} X2_i + \epsilon_i$, where we use our estimate of $\hat{\beta}_{21}$ as an estimate for the ATE.

Question

Considering that an estimand is a "quantity to be estimated in a statistical analysis", which of the following assertions is false and why?:

  • $\beta_{11}$ and $\beta_{21}$ are different quantities being estimated and thus are different estimands
  • ATE is the only estimand, with $\hat{\beta}_{11}$ and $\hat{\beta}_{21}$ being two estimates for it

(I know that the relationship between an estimand and its estimator can be fully arbitrary. For instance I could roll a die and have the result be the estimator for the average height of people in my country. It would be a useless estimator, but an estimator nonetheless. My question is about whether an estimand is uniquely defined in such context or if it is decidedly ambiguous. The motivation is thinking about its consequences in model averaging, for example)

$\endgroup$
11
  • $\begingroup$ I believe it is answered in detail in the thread linked as a duplicate. TL;DR estimand is the quantity you estimate, estimator is the method used for estimating it, and estimate is the result produced by the estimator. $\endgroup$
    – Tim
    Nov 9, 2021 at 14:07
  • $\begingroup$ I think the duplicate does not answer this question. The crux of the issue is exactly that the definition of estimand as 'the quantity you estimate' is sufficiently vague, that in some frameworks this issue has to be compensated with further distinction between a theoretical estimand (like the ATE) and an empirical estimand (see journals.sagepub.com/doi/abs/10.1177/00031224211004187). I don't think this approach is standard and hence my question. On reading the duplicate maybe what is confusing is calling the beta hats estimators instead of estimates, which will I edit. $\endgroup$
    – Kuku
    Nov 9, 2021 at 15:54
  • $\begingroup$ I'm not sure what is your question? The definition is broad since it is a very general concept. Regarding question from your question: every parameter you estimate is estimand. $\endgroup$
    – Tim
    Nov 9, 2021 at 16:08
  • 1
    $\begingroup$ It is answered since it defines the concepts. Estimand is the thing you estimate. Same as divisor is the thing you divide by. $\endgroup$
    – Tim
    Nov 9, 2021 at 16:17
  • 1
    $\begingroup$ They both estimate the parameters of the regression model. You can have multiple estimators for the same thing. As you noticed, anything could be an estimator and you can have multiple possible estimators for the same thing and this is a common case in statistics to have more than one possible estimator (e.g. you could estimate regression using OLS, MLE, or Bayesian approach). $\endgroup$
    – Tim
    Nov 9, 2021 at 18:26

2 Answers 2

2
$\begingroup$

The ATE is an estimand involving unseen potential outcomes and is defined at $E[Y^1-Y^0]$, where $Y^1$ and $Y^0$ are the potential outcomes under treatment and control. Under the main causal assumptions, the ATE is equal to $E[E[Y|A = 1, V]-E[Y|A=0, V]]$, where $V$ is a valid adjustment set. Let's call $E[E[Y|A = 1, V]-E[Y|A=0, V]]$ the average marginal effect (AME), which doesn't have a causal interpretation except when the assumptions that make the ATE equal to the AME are satisfied. The AME is also an estimand, but it doesn't require specific causal assumptions to be true to estimate it. It is possible there are multiple sets $V$ that make the AME with respect to $V$ equal to the ATE.

When a model is parameterized in a certain way, it is possible for a parameter in that model to correspond to the AME under certain assumptions that link the model parameter to the estimand.

Consider the following estimands:

  • $AME_{12} = E[E[Y|A = 1, X_1, X_2]-E[Y|A=0, X_1, X_2]]$
  • $AME_2 = E[E[Y|A = 1, X_2]-E[Y|A=0, X_2]]$

Under DAG 1, $AME_{12}$ is equal to the ATE, and $AME_2$ is a confounded association between $A$ and $Y$. Under DAG 2, $AME_2$ is equal to the ATE, and $AME_{12}$ is the direct effect of $A$ on $Y$ not through $X_1$.

Consider that the true outcome model is linear in the covariates and treatment and that there is no interaction between the treatment and covariates (i.e., so that your first model perfectly describes the data-generating process, which is consistent with both DAG 1 and DAG 2). Under this assumption, in your first model, $\beta_{11}$ is equal to $AME_{12}$, and in your second model, $\beta_{21}$ is equal to $AME_2$.

So, under certain assumptions, a $\beta$ is equal to an AME, and under additional assumptions, the AME is equal to the ATE. So what quantity does $\hat{\beta}_{21}$ in an OLS regression correspond to in your second model estimate? It estimates $\beta_{21}$. How you interpret that with respect to an estimand depends on the assumptions you make that link $\beta_{21}$ to the estimand you desire.

It is possible to estimate the AME using a different method, e.g., inverse probability weighting (IPW). IPW does not involve specifying a regression model for the outcome; therefore, the IPW estimand does not necessarily correspond to $\beta$ in any regression model. In this way, even if we aren't willing to make the assumptions that would link $\beta$ in some regression model to the AME, we can still use IPW to estimate the AME. This is important because we can describe the AME as an estimand separate from $\beta$, which hopefully clarifies that $\beta$ and the AME are not the same estimand except when specific assumptions link them. Similarly, IPW does not target $\beta$ except when $\beta$ is equal to the AME by virtue of the linking assumptions.

Let's wrap it up: the ATE, $AME_{12}$, $AME_2$, $\beta_{12}$ and $\beta_2$ are all potential estimands. The OLS estimator of $\hat{\beta}_{21}$ is generally unbiased for $\beta_{21}$. Under certain assumptions, $\beta_{21}$ may be equal to $AME_2$. Under additional assumptions, $AME_2$ might be equal to the ATE. If these assumptions are all true, then you can say $\hat{\beta}_{21}$ is an unbiased estimator of the ATE. But, again, whether that is true depends on the assumptions linking each quantity to the next; some of those assumptions are encoded in the DAG and others in the form of the outcome model.

$\endgroup$
13
  • 1
    $\begingroup$ If two sets of variables each form valid adjustment sets, then the AMEs based on the two sets will both correspond to the same estimand, the ATE. For example, adding an instrument to a valid adjustment set will not change the estimand and will still be a valid adjustment set. Model averaging is useful when you don't know the form of the model but you know the variables that go into it. The estimand is the same but the models differ. $\endgroup$
    – Noah
    Nov 13, 2021 at 17:35
  • 1
    $\begingroup$ I want to estimate the ATE. Therefore it is my estimand. Assumptions tell me the ATE is equal to an AME. So the AME is also my estimand. Further assumptions tell me the AME is equal to a regression coefficient. So the coefficient is also my estimand. If any of those assumptions are not true, or if you take an estimation approach that doesn't rely on them, then what is and isn't your estimand may differ. For example, if I use IPW to estimate the AME, I'm not relying on the assumptions that link the AME to $\beta$, so $\beta$ is not my estimand, but the AME and ATE are. $\endgroup$
    – Noah
    Nov 13, 2021 at 17:38
  • 1
    $\begingroup$ "what is the estimand depends on knowing the ground truth" This si not what I meant. The definition of the ATE is $E[Y^1 - Y^0]$; it makes no reference to adjustments sets. Under strong causal assumptions (the "ground truth" as you say), the ATE is equal to the AME with a specific adjustment set. Every causal inference procedure needs assumptions about the causal structure of other variables in the system (i.e., to be able to determine a valid adjustment set). But the definition of the estimand does not depend on these assumptions. $\endgroup$
    – Noah
    Nov 15, 2021 at 16:33
  • 1
    $\begingroup$ That is exactly right. Of course typically the purpose of estimating $\beta$ is with the hope that $\beta$ has a useful (i.e., causal) interpretation. Even if you have the right adjustment set, $\beta$ might not correspond to anything useful (e.g., the AME) depending on how the model is parameterized. For example, when $X$ and $X \times Z$ are predictors in a model, the coefficient on $X$ is generally not equal to the AME. That doesn't mean it isn't possible to construct an estimator of the AME from the model, but it just means $\beta$ isn't that estimator. $\endgroup$
    – Noah
    Nov 15, 2021 at 16:53
  • 1
    $\begingroup$ Your comment above is correct. Thank you for the edits and my apologies for the sloppy writing. The expectation can be taken over whatever your target population of interest is. I was considering the AME in a broader sense which is not always in the context of a linear model. $\endgroup$
    – Noah
    Dec 1, 2021 at 14:31
0
$\begingroup$

I prefer to view it according to your second bullet point. You have clearly identified the estimand of interest at the very top. The two different models can be seen as sensitivity analyses, each one relying on an unverifiable assumption. This is related to the ICH E9 addendum on estimands for clinical trials. In that setting we would define a particular population-level treatment effect of interest as the estimand. Whether we make a missing at random given treatment assumption, or missing at random given treatment and age assumption, or a missing not at random assumption for post-baseline confounding, these are all different missing data assumptions one could use in pursuit of the originally defined estimand.

$\endgroup$
3
  • $\begingroup$ What would be needed to suggest that any kind of estimate for the ATE is not a sensitivity analysis, but the main analysis? In other words, what alternative estimate would not depend on unverifiable assumptions? $\endgroup$
    – Kuku
    Nov 13, 2021 at 11:00
  • $\begingroup$ In practice there is no one who can tell us the "true" model for how confounding was introduced, we can only posit one. Any one model can be deemed by the experimenter as primary and the rest as sensitivity. This is done in clinical drug development for the purposes of decision making so that the decision maker does not just pick the model that produces the results that they were hoping to see. The experimenter does not necessarily have to label one model as primary and the rest as sensitivity $-$ each model can be a sensitivity for the others. $\endgroup$ Nov 13, 2021 at 14:24
  • $\begingroup$ Ideally all models will produce conclusions that point in the same direction. $\endgroup$ Nov 13, 2021 at 14:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.