I was recently considering the following: Suppose we have an experimental set-up where we have collected observations over thousands of locations (S) before and after treatment (T). Further, we have external information about the status of a continuous variable ($H$), which identically affects $Y$ pre and post-treatment and can vary with treatment. Is it possible to infer the direct effect of treatment on a single location (assume we have independent replications at each location), $S=s$, even when $H$ is changing? In the language of mixed models, I want conditional models for each $s$.
A bit new to the language of causal inference, mediation, and so on, so apologies for any confusing phrasing. But from what I can tell, this doesn't seem exactly like mediation analysis because $H$ has been measured pre and post-treatment ...
The total effect at s, as I understand would be defined as:
$$ TE_s=E[Y|T=1,H=1,S=s]-E[Y|T=0,H=2,S=s] $$
However, it is not clear if the "direct effect" could be estimated from the data? What we fundamentally want is a quantity that is independent of the level of $H$, but since $H$ varies with treatment as well, it is not clear that the direct effect has a straightforward proposal.
One way would be to say that the direct effect is the difference in the expected value of the observations given that $H$ is constant, but then what should $H$ be set to? None of this is immediately clear to me ...
I had a strange thought recently while pondering this question, and thought of the following construction. Suppose now we consider again the total effect at $S=s$, but somehow from our entire dataset pool, we find other locations with identical (similar) levels of $H$ at both the pre and post-treatment levels. We then take those samples and construct the following set:
$$ K=\{s|(H=1\cap T=1)\cap(H=2\cap T=0)\} $$
If all the locations in $K$ are not directly affected by treatment, but could show effects attributable to $H$, then I estimate the marginal treatment effect over all units $K$, $TE_K$, and compute $TE_s-TE_K$. Would this estimate be meaningful?
The rough idea I had was that if I somehow could derive the average change in $Y$ from all positions where the mediating variable are identical to the position I am interested in, then the effect of $H$ could be removed.
In the language of a standard GLM (with identity or log link) where I let $f$ be some arbitrary function that specifies the effect of the $H$ on $Y$.
$$ TE_s=g(E[Y|T=1,H=2,S=s])-g(E[Y|T=0,H=1,S=s])=\beta_T+f(h=1)-f(h=2) $$
Under the assumption that the units in $K$ do not change with treatment, and that $f$ is shared between all units:
$$ TE_K=g(E[Y|T=1,H=2])-g(E[Y|T=0,H=1])=f(h=1)-f(h=2) $$
It seems like then:
$$ TE_s-TE_K=\beta_T+f(h=1)-f(h=2)-(f(h=1)-f(h=2))=\beta_T $$
If the above holds, then the main challenge would be to find the set $K$ in estimating the direct effects independent of the changes in $H$. Is this a valid interpretation?
A few alternatives I quickly thought of while thinking about the problem:
- Could I chose to somehow estimate $f$ semiparametrically, and then remove the effects of $f$ from the analysis?
- Could I run a mixed or fully Bayesian model and estimate the treatment effects at each site in a single model with random slopes for the treatment effect, while incorporating a smoothing term to estimate $f$.
Would these also work at removing the indirect effect attributable to $H$?
