Analysis strategy for rare outcome with matching

Question

I'm working with a dataset of ~100,000 individuals where ~500 (0.5%) individuals received treatment.

I have several continuous and count outcomes for all observations that I would like to compare between treated and untreated. It would be important for the analysis to match individuals on several characteristics (that could be binary, continuous or categorical).

I'm working with Stata.

I was brainstorming several possible scenarios that include:

1. stratified analyses of treated and untreated

2. treating it as case control study and attempting to match 'controls' to all my 'cases' where criteria of match allow. Then moving forward with analysis appropriate for that set up (conditional logit would work for binary outcomes.. not sure about continuous and count ones..)

3. Treatment-effects estimation, perhaps using propensity-score matching (not sure if and how it is possible to include categorical variables though..)

What analysis would be most appropriate for such dataset?

Answer 1

Removing good data from an analysis is scientifically suspect in my humble opinion, and naive matching methods are inefficient. It may be very easy to adjust for patient characteristics using ordinary regression models, paying attention to linearity assumptions etc. Of course it is a good idea to look at overlap in covariate distributions across treatment groups to see where assumptions of no interaction between treatment and characteristics might be on shaky ground and untestable.

Answer 2

Based on the comments and the availability of such a large control group, I would probably advise to do in a step first exact matching on age groups and sex, and perhaps common disease groups. Hereby, you built different strata. In a second step, you can apply propensity score matching to ensure that treatment and control group are as balanced as possible with respect to the remaining observables.

You can do this apparently using the psmatch2 package for Stata (I have used that package only briefly out of interested).

A code example is given in the help file:

    g att = .
    egen g = group(groupvars)
    levels g, local(gr)
    qui foreach j of local gr {
            psmatch2 treatvar varlist if g==`j', out(outvar)
            replace att = r(att) if  g==`j'
    } 


    sum att

See here for further information

http://repec.org/bocode/p/psmatch2.html

You should -- of course -- verify that there is enough overlap between treatment and control group within each strata.

* Update: Response to the comment of Frank Harrell *

Why I argue for matching:

It is a trade-off between, on the one hand, balancing covariates as close as possible between treatment and control group, and, on the other hand, removing data (what Frank Harrell emphasized).

It is clear that the estimator becomes in a first step less efficient if you remove data, and you should justify ignoring data. But radek has asked for matching approaches and I agree that this is a good idea.

Matching avoids to some extent "extrapolation bias" if the covariate distribution differs between treatment and control group. You drop observations which give you few or any information about the treatment effect because their covariates are very far away from the sample.

Many prominent researchers therefore recommend matching or subclassificatioon plus regression.

See Imbens & Rubin, Ho et al., Imbens & Wooldridge

Answer 3

The propensity score (PS) is a balancing score indicating the probability of treatment assignment conditional on observed baseline characteristics. In a randomized controlled Trial (RCT) the PS is known. Estimation and application of PS therefore, mimic some of the particular characteristics of an RCT and therefore are method of choice to estimate the treatment effect in an observational study, provided that you have no unmeasured confounders and all subjects have a non-zero probability of receiving Treatment.

You can estimate the PS using logistic regression or generalized boosting methods and therefore, include all observed (continous and categorical) baseline variables. Notably, you do not include the outcome parameter. (There is an ongoing debate, which variables to include, however, Austin postulates it is safe to include all observed baseline variables https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3144483/). It is important to underscore that you do not aim to find the best "predictive model" but instead the model that balances your covariates best. After application of the PS using stratification, matching, inverse probability Treatment weights or covariate adjustment (all of them have pros and cons and are somewhat dependent on your data), you can test the treatment effect on your outcomes.