I have a group of physicians that want to test whether or not the education of staff and making access to a particular medication/device more available to a certain population is statistically different than the previous standard of care. They are measuring 3 months of data at baseline, then implementing education and access changes and then measuring 3 months of more data. The intervention takes a couple of months to educate and get the access up and running. During the first month they are looking at patients that had a particular event in the previous 26 months and whether or not they had this particular medication/device in place, during the second month the same collection takes place, and then during the third month the same data collection takes place. This is all taking place using Electronic health records. From my understanding one patient could be counted in all 3 months if they met the criteria and patients could fall off during the second month if they are outside the window and be replaced with the same number of new patients that meet the criteria. This would result in the numbers potentially looking the same even though it is different patients.

The data collection after intervention would be the same process. You could also have patients in the post intervention data collection that were in the pre as well if their event was recent. I am unsure whether to look at it on a month by month basis or look at it from a 90 day window and count all those patients then look at a 90 day window after the intervention. Thanks for any input.


Based on your particular experimental design, and the way the patients are sampled, I would look at it on a month by month basis, since it is possible that the same patient is sampled in all three months. Using 90 day to 90 day would effectively weight the sample towards the patients who have had the condition for the longest time. Eliminating the repeated sampling from the data-set has the same effect as fundamentally changing the sampling scheme.

Your physicians seem to be interested in the change of the number of patients who have the device over time, (gleaning this from the way this study is designed), and if this is true, then a month by month will be far more informative than simply looking at before/after. Using month by month can show change in use of the device over time both before and after the change is implemented.

Since you tagged the question with hypothesis testing, I'll try to address that more specifically here. A good null hypothesis, or H_0, would be that the rate of device usage is the same across all 6 months. If you find that you cannot reject H_0 for the first 3 months, then you have evidence to support that the change had an impact if you can reject H_0 for the second 3 months. If you do reject H_0 for the first 3 months, then you have a rate of change in device usage in your base case. This can be your new H_0 (i.e, the rate of change in device usage is X, based on the base case). You can show that the change you implemented had an impact if you can reject that H_0 for the second three months. (i.e, the rate of change in device usage is Y during the second portion, which is different from X with p = 0.04).

Hope this helps.

  • $\begingroup$ Thank you for your input on this. Will try to get this set up in R. $\endgroup$ – Brad Jul 24 '19 at 18:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.