1
$\begingroup$

Imagine that I am trying to simulate a data generating process where I make the following assumption about $Y$:

$Y$ = $X$(0.15) + $Z_1$(0.23) + $Z_2$(0.08) + $Z_3$(0.19) + $Z_4$(0.05) + Error

Also, consider that I am interested in several estimators and I am trying to find which estimator recovers the defined treatment effect of 0.15 the best. However, $X$ is confounded by several variables, so I adjust for them accordingly.

What I understand well is that, for variables that I assume to be confounders, I need to simulate effect sizes for each confounder on the treatment and outcome. For example, if $Z_1$ has an effect of 0.23 on $Y$, I would also need to simulate an effect of say, 0.04 on $X$, otherwise, it would not be a confounder.

However, in more complex data generating processes, the confounders may impact the value of other confounders themselves. That is, it may be inappropriate to define each confounder as a distribution completely exogenous to the other confounders of interest.

For example, rather than saying that $Z_1$ is just a normally distributed variable with a mean of $\mu$ and a standard deviation of $\sigma$, I may also say that $Z_2$ has an effect of 0.33 on $Z_1$ and that $Z_3$ has an effect of 0.07 on $Z_1$.

For the goal of this hypothetical simulation (testing different estimators to see which recovers the treatment effect the best), is it necessary to define the effect that each confounder has on the other? Or, as long as I specify the effect that each confounder has on the outcome and treatment, is the simulation analysis good to go?

On the one hand, I see benefits in fully detailing the assumed data generating process for all variables in the system. However, on the other hand, I see problems with me embedding further assumptions into the analysis for effect sizes that are not of interest (i.e. I would have no actual interest in knowing/pondering how $Z_3$ might impact $Z_1$, $Z_2$, $Z_4$, etc.).

$\endgroup$

3 Answers 3

1
+100
$\begingroup$

If I understand your question correctly, you are asking how to model confounding in general when a "treatment" variable $X$ and an "outcome" variable $Y$ may be embedded in an arbitrary larger graph with confounding variables $Z_1, \dots Z_m$. Given your equation, it seems to me like you are assuming linearity of the structural equations.

If you had two confounders $Z_1$ and $Z_2$ then, given linearity, in terms of estimating the avearge treatment effect this would be the same as one confounder with weights and a distribution that reflect the combined effect of both. Similarly, if there were multiple confounders affecting one another, one could still subsume them into one single confounder with a strength and distribution reflecting the total arising from their interaction. So to investigate the effect of different confounding, it should suffice to change the weights and distribution of a single confounder.

With that being said, your estimators should recover the true causal effect regardless of what confounders you build in, provided your model is well specified and adjusted correctly, and given sufficient samples. I think it's a great idea to run some simulations and I would definitely play around with different confounder set-ups to build intuition, but when it comes to choosing an estimator, I would strongly suggest looking at their theoretical properties.

$\endgroup$
0
$\begingroup$

if you allow me to write a few thoughts on your problem:

first and foremost, what is usually done in this kind of analysis (i.e., multiple parameters jointly influencing some outcome), is that you estimate intercorrelation between particular parameters. You can use just Pearson, ICC, or VIF for that. This way you can exclude highly intercorrelated variables even before designing you model. With simulation things can be a bit more complicated, although you can always split simulation into: generating the data, checking intercorrelations, and running the model.

That is of course, if I got your problem right.

What I understand well is that, for variables that I assume to be confounders, I need to simulate effect sizes for each confounder on the treatment and outcome. For example, if Z1 has an effect of 0.23 on Y, I would also need to simulate an effect of say, 0.04 on X, otherwise, it would not be a confounder.

However, in more complex data generating processes, the confounders may impact the value of other confounders themselves. That is, it may be inappropriate to define each confounder as a distribution completely exogenous to the other confounders of interest.

Why do you consider X as being mathematically different from Zx? From the strictly technical perspective these are all linear regression parameters and they all can influence each other. It does not matter whether some of them are clinically relevant, or are confounding variables.

Please also note that a lot of your rationale depends on how you define a "confounder" exactly. What definition are you using, from which interpretation/handbook? In my opinion there are two options:

  1. A counfounder Z is a variable/phenomenon that explains the variance of the Y, while the variance of Y is expected, and should be, influenced only by X.

  2. A counfounder Z is a variable/phenomenon that explains the variance of the Y, while the variance of Y is expected, and should be, influenced only by X; and Z explains the variance (is correlated with) X.

See the difference? I can imagine scenario in which two events totally unrelated to each other are jointly influencing some other variable. Let me use the following example: a patient is being stressed out by his boss at work, and by his spouse at home. The boss and the spouse are unrelated, but they both, jointly or individually, can contribute to the patient developing a depression. Now, which one of the two is a confound? It depends on what are you studying. If you are interested in the influence of work-related stress to developing depression, then home situation is your confound. If you are studying marriage relationships, then what is happening at work is the source of your confounding variables.

Now, the example of partly related/correlated X and Z influencing Y would be if the boss (X) and a co-worker (Z) would be stressing out the patient enough to get him depressed (Y). Why? Because the co-worker (Z) is also stressing out the boss (X) (not only our patient), and this is in part the reason why X is contaminated by Z. If you remove a confound Z, then X would explain Y better, but bear in mind that Y can still be the same, or can change, only an experiment would tell (or a specific simulated dataset). Please also note, that in this situation it is very hard to tell, whether it is not also the other way around, i.e., boss (X) is stressing out the co-worker (Z), and they both influence patient (Y). At some point, the theoretical deliberations are not enough, and you have to head out to practice to disentangle different phenomena.

as long as I specify the effect that each confounder has on the outcome and treatment, is the simulation analysis good to go

I would say yes.

However, on the other hand, I see problems with me embedding further assumptions into the analysis for effect sizes that are not of interest (i.e. I would have no actual interest in knowing/pondering how Z3 might impact Z1, Z2, Z4, etc.).

It depends on the problem. If you want to provide more cognitive explanation to the simulation process, you can model these influences, but I can imaging a different goal for you simulation to just estimate the influence of X and Zx treated as separate variables (bear in mind the first paragraph of my answer). Interestingly, in one of your recent analyses, when we removed inter-correlated variables, then our model performed worse. Does that mean that the intercorellations were artificially improving the accuracy, or that despite intercorrelations model was getting some important information from more number of parameters? Only an attempt to apply the model in practice (e.g., prospective clinical trial) would provide an answer to this question. I would not trust only simulation, unless there was a clear mathematical/logical proof that I would be able to understand. Otherwise I would go with empirical experiment, because its outcomes are more cognitively available.

To further comment on you immediate methodological/statistical issue, concerning taking into account how particular parameters influence not only the outcome, but also each other, I would advise you to look at your problem, even a simulated one, from the practical perspective of some example research question, it helps a lot to get the right answers, even on the theoretical level.

$\endgroup$
0
$\begingroup$

If you wish to solely simulate under a tight set of conditions then as @scriddie says you can condense the confounders into a simplified solution. If however you wish to explore how the simulation unfolds as it is subjected to different background conditions then it may be useful to include any confounders that can be relevant to the scenario.

See this paper simulating validation of a real world model in different scenarios Confounders have the potential to influence the model considerably, but need to be modelled in a way that allows this to be explored.

This approach is only useful if the confounders have their own DGP coded and can their properties can be set independently. If the confounders are defined solely on their interrelationship (e.g. a fixed correlation matrix, or a specified mathematical formula) then there is no value in undertaking this complexity.

Only make your model as complicated as you need to, so that you can interpret what it is telling you.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.