Role of regression model fit in causal analysis

Question

When analysing causal questions, we use DAGs that give us covariates needed for modelling. But another time we assess model fit to get the best prediction. These two approaches have different purposes and are mostly used separately (?). But is there also a place for their simultaneous use?

For instance, we have a nested data, and we would like to analyse temporal trends on an outcome.

Minimal sufficient adjustment sets for estimating the direct effect of year on outcome:

covariate, state

Though the DAG gives us model structure, we still have multiple options for specifying our model. For example, we can use hierarchical modelling if we know that there were regionally different baselines and/or varying temporal trends.

outcome ~ covariate + state
outcome ~ covariate + (1 | state)
outcome ~ covariate + (covariate | state)

Thus, would it be a case in causal analysis where we should also assess the model fit? Meaning that the published model was chosen according to the DAG and model fit comparison?

And is there also another dimension - interpretation. When reporting predicted country-wide trends (not regional trends), isn't it so that the first additive model only gives us the temporal trend for one state (state and covariate variables needs to be fixed on one value)? In contrast, hierarchical models' temporal trends predictions are somewhat done for all states at once? If my understanding is true, the first additive model is poor for reporting country-wide trends?

Answer 1

A DAG is a non-parametric model of the causal relations among a set a variables. The DAG tells you that covariate and state should be adjusted for, but it tells you nothing about how to adjust. Covariate adjustment is one approach, and stratification is another. Here you are proposing three covariate adjustment models:

outcome ~ covariate + state
outcome ~ covariate + (1 | state)
outcome ~ covariate + (covariate | state)

The DAG does not help you choose which one to use. Each model represents different parametric assumptions, none of which the DAG can inform us about. There are many considerations for choosing among these models. A few are:

• how mahy state's are there ? If too few then the first model is the only option.

• Does covarate vary within levels of state ? If not then the third model is not appropriate. If it does, then the 3rd model may be appropriate provided that the model is supported by the data (and random slopes often are not - despite being clinically/theoretically appropriate)

• Also note that, for the exact same reasons, the DAG can't tell you whether you should include nonlinear terms such as quadratic, cubic, or splines; nor can it help determine any transformations that might be needed