I am reaching out to the Cross-Validated statistical community seeking suggestions on a challenging problem on which I'm working. I've been asked to look into a problem related to electronic submission of certain types of doctor's orders on patient outcomes compared to paper submission. Specifically we are trying to understand if the movement from paper to electronic doctor's orders (or a greater proportion of electronic orders compared to paper orders) results in reduced in-patient mortality and decreased hospital length of stay. The theory goes that electronic orders can be executed faster and algorithms built into the electronic ordering process might reduce medical mistakes (e.g. medication errors, transcription problems, etc.)

The decision of whether or not to submit an electronic order is not random, but is essentially at the discretion of the provider. Given the non-random assignment of orders, I'm worried about large sources of selection bias and confounding. After some thought, I figured I'd use a propensity score approach to minimize selection bias, but this approach wouldn't be as straightforward as it usually would be. Usually when using propensity score analysis, you estimate the single probability/propensity of getting the treatment given baseline covariates and then carry out the rest of the analysis using the propensity score in matching, weighting, subgrouping, or adjustment. In this case, I have several treatments all of which aggregate into some net effect of the patient's outcome (shorter/longer length of stay or death/non-death).

I thought it might be possible to estimate many propensity scores, each at the individual order level, and then simply take the mean of the propensity scores across all of the orders that occurred during a patient's hospital stay to obtain an overall average propensity to receive an electronic order for the patient. This would then give me a single propensity score that I could use to carry out the analysis at the patient level. But this seems problematic to me for a number of reasons. I think the process of aggregating could drown out certain order propensities that might have a large positive or negative impact on outcomes. This approach seems problematic.

So, I wanted to see if anyone has used propensity scores in a similar manner or has any other suggestions on how I might go about this analysis and adjust for selection bias given my one-to-many problem of several orders and one outcome. Are there types of hierarchical propensity score approaches that others have used similar to this? Any thoughts, suggestions, or references would be much appreciated.


One approach you might consider is to narrow the focus of your study. Instead of using "medical outcomes" as your dependent variables, you might have a simpler task if you use "medical mistakes" as your dependent variable(s). This approach is common in quality improvement analysis in manufacturing and services. The causal path is shorter, which should simplify your analysis considerably and probably strengthen your conclusions. I assume that there are published papers that investigate the relationship between various mistakes (e.g. transcription errors) and health outcomes. You can draw on those results to extend and augment your results.

The challenge, of course, is how to get data on the rate of medical mistakes of various types. If I were in your place, I'd make a strong plea for that data to be collected.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.