hypotheis testing in observational study with propensity score matching to reduce confounding

Question

in observational studies, many people use propensity score matching to reduce confounding (measured co-voriates) between two groups (cohorts). But due to some unobserved confounding co-variates (not measured), there is usually some residual confounding, which makes it impossible to draw causual conclusion for the treatment effect with hypothesis testing.

Then, 1. how to estimize the study size (sample size) for an observational study (with two cohorts), without utilizing these hypothesis testing based methods as for the randomized clinical trials? 2. other than the descriptive analyses, what comparison analyses can we do for observational studies? how do we evaluate the effect of the treatments from the two groups?

Answer 1

Residual confounding doesn't completely invalidate all of hypothesis testing for treatment effects. All it does is increase the bias of your treatment effect estimate. That bias may be inconseqential; in a well-designed study, enough variables would be collected to plausibly eliminate residual confounding. If it's consequential, the estimated effect doesn't become uninterpretable; it's just not interpretable as a causal effect. It may be interpretable as something like a causal effect, i.e., a relationship after controlling for many sources of confounding. How you interpret an estimated effect has nothing to do with how you compute power or plan sample size calculations. Also, whatever statistical method you use to estimate the effect may give an unbiased estimate of a non-causal relationship. Choosing to focus on this relationship doesn't invalidate the use of hypothesis testing to assess whether your data support this relationship being different from null. So, to answer your specific questions:

1) You use the same methods as for a randomized study while accounting for the fact that you may need a larger sample to eliminate confounding (e.g., because in matching you need a larger pool of controls or you need more units to support more variables in a regression model). If you weren't intent on estimating an effect that is interpretable as causal, then you can just use standard procedures. This is what all researchers do for all studies that aren't causal in nature.

2) There are some methods that allow you to estimate a treatment effect in the presence of unmeasured confounding. You can just use those. You can also create bounds for the true causal effect based on the possibility of an unmeasured confounder. You can also estimate a causal effect and assess the sensitivity of the substantive conclusion to unmeasured confounding. There is a huge literature on unmeasured confounding and omitted variable bias. We don't have to completely abandon an entire statistical and inferential framework just because there are a few variables we can't control for.