10
$\begingroup$

I working with observational data and defining assumptions for DAG seems to be more complex than often in examples provided in textbooks. For me, it would be much easier to just skip DAG part and condition for everything, and probably there will be no problem in publishing. However, I like the idea of being explicit with my causal assumptions under methods.

I'll give a simplified example with only two predictors, and thus it would be easier to follow.

Background Information

  • a crude analysis shows very clear multifold regional disparities in income between people from different towns
  • variables or nodes like sex and age differ between towns (p < 0.05), indicating a need for adjusted analysis
  • I have done different uni- and multi-level models with all kind of combinations of predictors and the result of regional disparities always holds.

DAG

Enter image description here

Blue arrows seem to be okay for me; however, relationships between towns and sex/age are quite hard to define. I'll bring few, maybe stupid, examples

  • town may be the cause of different sex distribution by offering more jobs for one sex (e.g. men and mining towns)
  • sex may be a reason to change residence (e.g. local policies discriminate women and they move to another town)
  • town may be polluted and shorten our expected living years (age)
  • age may be a reason to change residence (e.g. moving to another town to go to university)

As you see, causal assumptions can be unidirectional (red, green) or bidirectional (orange), or is it even more reasonable to show them as undirectional (no arrows) (black)?

Goal

  • As age and sex differed between towns, there will be a question about adjusted analysis. The goal is to use adjusted analysis to confirm the results of crude data analysis (to make them more bulletproof) - regional disparities between towns.

What would be the best way to achieve my goal?

For me, it seems that publishing the most conservative result would be reasonable since the result won't change with any adjusting.

What would be the most conservative adjusting?

  • adjusting for everything, age and sex, and even if they may partially act as mediators (unknown direction or bidirectional arrows)?
  • should I show a causal graph with undirected arrows (how should I name it then)?
  • should I show a causal graph with bidirectional arrows (still named DAG?)
  • am I right that undirected and bidirected arrows both make sex and age confounders due to opening a back-door path?

How would you solve and present this situation in your article?

$\endgroup$

3 Answers 3

12
$\begingroup$

Fist, I think it is good that you are using a DAG because it requires careful thought about causality, and this is often at the heart of modelling.

adjusting for everything, age and sex, and even if they may partially act as mediators (unknown direction or bidirectional arrows)?

One approach to this is to estimate the net effect for each variable that could either be a confounder or a mediator, and then adjust as appropriate. How you estimate the net effect is another question of course. You could also just make an assumption (and state the assumption in the paper). Another idea is to fit several models where the variables are treated as either mediators or confounders and report the results of all. Since you only have 2 variables, Sex and Age, this seems like a reasonable approach; it would mean fitting 4 models.

should I show a causal graph with undirected arrows (how should I name it then)?

I would not do this, as it makes the diagram ambiguous.

should I show a causal graph with bidirectional arrows (still named DAG?)

I would not do this either, if you are fitting 4 models, as it would be inconsistent with the modelling. Also, you can't call it a DAG if it has bidirectional arcs (a DAG is dorected by definition)

I would include 4 DAGs.

am I right that undirected and bidirected arrows both make sex and age confounders due to opening a back-door path?

Not really, if you are following DAG theory, because the presence of an arc with no direction means that the graph is not directed and therefore is not a DAG.

$\endgroup$
8
  • 2
    $\begingroup$ Your last point is not quite accurate in terms of notational convention. Bi-directed arrows are conveniently used in DAGs to symbolize open back-door paths due to unobserved confounders. I have updated my answer with respect to this point. $\endgroup$
    – persephone
    Commented Sep 9, 2020 at 10:32
  • 5
    $\begingroup$ @BobD in my opinion that's an abuse of notation, but I do take your point that some people do use that convention. But in any case there are no unobserved confounders here. $\endgroup$ Commented Sep 9, 2020 at 10:37
  • 1
    $\begingroup$ The only thing I can think of at the moment is a sruvey of other literature where the net effect has been estimated over time. With that number of variables which could be either mediators or confounders, plus the presence of (potentiallly) unmeasured confounding, unfortunately I am sorry to say that, to me, the situation looks quite hopeless for a regression model. Perhaps an SEM might be worth pursuing, but without a lot more detail I can't say. Please ask a new question, where you explain your study design, and give details of the data, about how to answer your research question(s) $\endgroup$ Commented Sep 9, 2020 at 11:25
  • 1
    $\begingroup$ Unless you can find evidence, or you are willing to make assumptions, about the direction of causality, there isn't much you can do with regression. There is no test to determine the direction of the causal effect when you only have cross-sectional observational data. Even with an SEM, you will need to create latent variables and then you will need to specify in which direction causation flows. $\endgroup$ Commented Sep 9, 2020 at 11:32
  • 2
    $\begingroup$ I side completely with Robert on these last points. And yes, I also agree that bi-directed arrows is not good convention! $\endgroup$
    – persephone
    Commented Sep 9, 2020 at 11:51
6
$\begingroup$

If you are not sure about the direction of the arrow, this is likely because you suspect (implicitly or explicitly) some potential confounding between the two variables. Hence, you should draw all plausible graphs and derive identifying assumption for each. For some you might reach the conclusion that your causal quantity of interest is not identifyable vis a vis available data, for others it might. With the DAGs you make clear under which causal assumptions a causal interpretation of your empirical estimand is internally consistent.

Generally speaking, the causal interpretation of an empirical estimand is based on the underlying causal model. That is, based on likely untestable assumptions. The DAGs are a tool to make this clear.

Bi-directed arrows are used in DAGs to indicate that there are unobserved back-door paths between two variables. You could also include this unobserved confounder explicitly, labelling it for instance $U$. This is just notational convention. However, assuming a bi-directed (or an unobserved confounder) changes, of course, the implications for identification.

$\endgroup$
2
  • $\begingroup$ Thank you so much! I have also seen bidirectional arrows in multiple articles. Am I correct, that I have to test the direction of each arrow using regression with my study data? If so, could you please check my response to Robert Long and maybe help to elaborate how to do this (my variables are categorical or continuous and non-normal and zero-inflated). $\endgroup$
    – st4co4
    Commented Sep 9, 2020 at 11:15
  • 2
    $\begingroup$ You cannot "test" causality in observational settings (and it is also often harder than assumed with experimental data). To repeat myself, you have to make assumptions: if you can draw a plausible DAG where you can identify the quantity youre interested and estimate an effect you can interpret it, internally consistent, as causal. But other causal models might be plausible too. DAGs make this transparent. For regression, you need to invoke further assumtions about functional form etc. DAGs in the form above are for identifying causal effects in a non-parametric setting. $\endgroup$
    – persephone
    Commented Sep 9, 2020 at 11:51
0
$\begingroup$

I think I can see some issues with your specific example that are easily corrected, but you have also struck upon some more general phenomena that I think merits some discussion as well.

Your specific example

I think the problem with the specific example you've given is that you're being a little too loose with

  1. the meanings of your nodes/variables
  2. the language around cause and effect

Meanings of nodes/variables

As an example of (1), I'll focus on the town of residence <--> sex relationship. In this scenario, "sex" is ambiguous because it can be applied at the level of an individual, but from the context you've given, I think you really mean to aggregate it at the level of the "town of residence". It might be better to call it something like "sex ratio" to make that more clear.

More importantly, "town of residence" implies some special coupling between a town and its residents. But there are aspects of a town that are independent of its residents (e.g. location, geography, weather, resources, etc). Why not be more general and have that variable simply represent the concept of "town (identity)"? Now it is plain to see that we cannot have sex ratio --> town, as there is no way the sex of a town's residents can influence its very identity; towns do not spontaneously change from one to another if their residents' sex ratio changes. However the culture of, jobs available in, and policies instituted by a town can certainly affect the sex distribution of its inhabitants through birth, death, and immigration, so you can have town --> sex ratio. To give a concrete example, at the time of writing, New York has a higher ratio of female:male (1.096) than Los Angeles (1.019). If you move from one city to the other you will observe the change in sex ratio. However, you cannot change New York into Los Angeles no matter how hard you manipulate the sex ratio or other attributes of the cities.

You may argue "well what if changing the sex ratio influences the culture and policies of the town" - that is a perfectly valid hypothesis; but the culture and policies of the town do not describe the town in its entirety, they are simply some subset of its attributes. So you can have relations like town --> sex ratio --> {culture, policies}.

Language of cause and effect

That previous fix alone should solve your problem, but I'll address point (2) as well about the language around cause and effect. For example, when you state

sex may be a reason to change residence (e.g. local policies discriminate women and they move to another town)

as evidence for the relation sex --> town of residence, I think this is actually better re-framed as the town creating discriminatory culture/policies, causing women to leave, i.e. town --> discriminatory culture/policies --> sex. Even though someone may feel that their sex is the "reason" they need to move (and I feel so sorry for anyone who may think this way!), an individual's sex can't be the cause of their need to move, because it cannot be changed (at least not very easily). The thing that changed and caused this individual to want to move is the creation of discriminatory culture and policy!

You may recall that we just said sex ratio --> {culture, policy} a couple paragraphs ago, but now we're saying {culture, policy} --> sex ratio is possible too... no, there is not a conflict here but you'll have to read the section below on "feedback loops" to see the resolution.

Some more general phenomena

Tightly coupled and missing variables

It may be the case that you have two or more variables that appear very tightly coupled in some way, but are actually related through a missing variable, like a common cause or influence. For example, consider several light bulbs connected together in series (e.g. like Christmas lights). As you may be aware, when wired this way, when one of the bulbs burns out, the whole thing stops working. When one bulb is on, they all are on; when one is off, they all are off; everything is highly correlated. If someone were to observe that the lights are off, they would not be able to tell which bulb is the one that caused the problem. Did bulb $b$ burning out cause the other ones to stop working, or was it because of bulb $b^\prime$? The direction of causality is very unclear here and the naive causal graph where nodes represent the on/off state of each bulb would be fully connected and bidirectional (and therefore not a DAG). However, if we could inspect and test each individual bulb, then we could in principle figure out which one was broken, and integrate that into our model of the on/off states of the lights. In this scenario you need more data, depicted by additional nodes in the DAG besides the on/off status of each bulb representing their working/broken state.

omitted variable graphs

Some common causes are a little simpler. For example, if all the bulbs in our string of lights are off, it could be that the power to the entire thing is off! This is usually a much simpler hypothesis, but we still need additional data, depicted by a node in the DAG representing power to the lights.

common cause graph

It's also possible, but less common, that two variables may share a common influence, or descendant node in the DAG that is implicitly being conditioned on. For example, a company's revenue might have two factors: sales volume and per unit price (pretend that these factors are totally independent), which are causally related via the "collider model" volume --> revenue <-- price. If you implicitly condition on companies with a specific level of revenue (for example by using data from just the Fortune 500), you'll likely find a negative relationship between volume and price (think about volume * price = revenue while holding revenue constant), which could lead you to conflicting causal conclusions like

  • high sales volume causes low prices because if it is so easy to produce more product, the barrier to entry is low and competition in the market is high, suppressing prices
  • high prices cause low sales volume because there is less demand at higher prices

In the collider model, both of these are wrong (though the second is actually true in real life) because there is no causal path between the two nodes.

Feedback loops

Sometimes you may have a scenario where there is some kind of feedback loop where A influences B, then B influences A again (or some even longer cycle like A -> B -> ... -> Z -> A), and this time it's not just an apparent cycle, you know for sure that it's real. This happens all the time in physics, economics, chemistry, etc:

  • the sun pulls on the earth via gravity, then the earth pulls on the sun, and so on back and forth forever, causing them to orbit each other
  • prices go up, which causes workers to demand higher wages; higher wages mean higher costs for firms, which causes firms to increase prices to maintain profit margins (i.e. inflation)
  • in a chemical reaction, reagents A + B interact to form C, but C will also spontaneously dissociate back into A + B, causing the relative number of each species to evolve until equilibrium is reached

The thing that all these examples have in common is the flow of time. When A influences B, it happens at a specific moment in time, and then when B influences A, that happens at a different, later, moment in time. The duration between these moments could be arbitrarily small in principle so for all practical purposes they could appear to be occurring simultaneously. Or if this feedback is happening slowly enough we could approximate this process with larger time steps (e.g. months or years might be reasonable in the inflation example). But the key point is that we can unravel an apparent relation like A <--> B into something like {A(t), B(t)} --> {A(t+1), B(t+1)}. Now you have sequential data that can be modeled by time series, differential equations, etc.

Summary/TL;DR

In conclusion, when you encounter a scenario like this where it is difficult to reason about the direction of causality, here's what I would advise you consider to get yourself unstuck:

  • are you being too loose with your definitions?
    • can you rename a node in order to make its meaning more precise? (e.g. sex -> sex ratio)
    • can you refine/generalize a concept that might be too specific? (e.g. town of residence -> town)
    • can you split a more general concept into smaller pieces or attributes?(e.g. town may have attributes {culture, policies, jobs, demographics, ...})
    • are you considering the appropriate level of aggregation (e.g. sex of an individual vs sex ratio of an entire town)
  • are you sure you're thinking about the direction of causality correctly?
    • how possible/practical is it to modify/manipulate a certain variable? (e.g. the sex of an individual vs. the sex ratio of a town)
    • if the state of A changes, but the state of B remains the same, any change in the state of C was potentially caused by the change in A, and A might not be influenced by B (i.e. A --> C <-- B might be a decent hypothesis)
    • think about "what would happen if I changed X?" and "how could X be changed?"
  • could your model have omitted variables that could explain things more fully?
    • could your variables have common causes you are not taking into account? (e.g. all light bulbs can be turned on/off by power, or missing variables for working/broken state)
    • could your variables have common influences that you are unknowingly/implicitly conditioning on? (e.g. volume --> revenue <-- price induces acausal relationship between volume/price when conditioning on revenue)
  • can you decompose the relationship at one moment into a sequence of events over time?
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.