Which test would be appropriate

Question

I am editing this question and including generated data that is similar to the data that I have from a clinic. We have Medical residents that want to look at a particular event and see what percentage of patients that are < 28 months from event and have a reversible device placed. Each month from Dec 2017 to Dec 2018 the Electronic health record was scanned looking back 27 months for particular event and device placement. Two interventions took place involving ease of access (April 2018) for particular device and educating staff (July 2018). They are looking to see if interventions had any significant bearing on increased usage. I am looking at how to set this up to test for significance. It was originally tested using t-tests of differences in average percentage of patients comparing 3 different months (Dec17-Feb18) to another 3 months (Oct18-Dec18). These 3 month blocks represent before and after interventions. The person that originally tested has moved and I was given the data. I think there is a flaw in the way the percentages were calculated. Each month had a lookback and usage was tallied, so from month to month the same patient was counted again if they met the condition of < 28 months from event. Plus the post intervention collection counted the same patients that were in the pre intervention months if they still met the < 28 month cutoff. I think either an interrupted time series or some pre post proportion test that did not count patients twice would be the way to go. Below is the data that simulates what is in the clinic. I had to do this for HIPAA reasons. Any help on this would be appreciated. The data is from r language.

set.seed(1)
id <- sample(100000:199999, 500, replace=FALSE)


Event_Date <- as.Date("2015-10-01") + 
  runif( 500, 
         max=as.integer( 
           as.Date( "2018-12-31") - 
             as.Date( "2015-10-01")))

sdf <- data.frame(id,Event_Date)
sdf <- sdf[order(as.Date(sdf$Event_Date)),]
rownames(sdf) <- NULL

fdr <- rbinom(n=396, size=1, prob=0.13)
sdr <- rbinom(n=40, size=1, prob=0.155)
tdr <- rbinom(n=64, size=1, prob=0.175)
sdf$Device <- c(fdr,sdr,tdr)



monnb <- function(d) { lt <- as.POSIXlt(as.Date(d, origin="1900-01-01")); lt$year*12 + lt$mon } 
# compute a month difference as a difference between two monnb's
mondf <- function(d1, d2) { monnb(d2) - monnb(d1) }

sdf$Dec17Scan <- ifelse((mondf(sdf$Event_Date,"2017-12-31")<28 & mondf(sdf$Event_Date,"2017-12-31") >= 0),1,0)
sdf$Jan18Scan <- ifelse((mondf(sdf$Event_Date,"2018-01-31")<28 & mondf(sdf$Event_Date,"2018-01-31") >= 0),1,0)
sdf$Feb18Scan <- ifelse((mondf(sdf$Event_Date,"2018-02-28")<28 & mondf(sdf$Event_Date,"2018-02-28") >= 0),1,0)
sdf$Mar18Scan <- ifelse((mondf(sdf$Event_Date,"2018-03-31")<28 & mondf(sdf$Event_Date,"2018-03-31") >= 0),1,0)
sdf$Apr18Scan <- ifelse((mondf(sdf$Event_Date,"2018-04-30")<28 & mondf(sdf$Event_Date,"2018-04-30") >= 0),1,0)
sdf$May18Scan <- ifelse((mondf(sdf$Event_Date,"2018-05-31")<28 & mondf(sdf$Event_Date,"2018-05-31") >= 0),1,0)
sdf$Jun18Scan <- ifelse((mondf(sdf$Event_Date,"2018-06-30")<28 & mondf(sdf$Event_Date,"2018-06-30") >= 0),1,0)
sdf$Jul18Scan <- ifelse((mondf(sdf$Event_Date,"2018-07-31")<28 & mondf(sdf$Event_Date,"2018-07-31") >= 0),1,0)
sdf$Aug18Scan <- ifelse((mondf(sdf$Event_Date,"2018-08-31")<28 & mondf(sdf$Event_Date,"2018-08-31") >= 0),1,0)
sdf$Sep18Scan <- ifelse((mondf(sdf$Event_Date,"2018-09-30")<28 & mondf(sdf$Event_Date,"2018-09-30") >= 0),1,0)
sdf$Oct18Scan <- ifelse((mondf(sdf$Event_Date,"2018-10-31")<28 & mondf(sdf$Event_Date,"2018-10-31") >= 0),1,0)
sdf$Nov18Scan <- ifelse((mondf(sdf$Event_Date,"2018-11-30")<28 & mondf(sdf$Event_Date,"2018-11-30") >= 0),1,0)
sdf$Dec18Scan <- ifelse((mondf(sdf$Event_Date,"2018-12-31")<28 & mondf(sdf$Event_Date,"2018-12-31") >= 0),1,0)

Thanks for any help guiding what test to use.

Answer 1

I would create a dummy variable that is equal to 1 for all values after the "treatment" and is equal to 0 for all values before the treatment. It's sloppy, but you can collapse across the months by taking the average value of the "Pts < 27" column and transforming that as a variable called percent_with_procedure.

Then run a regression of the form:

lm(percent_with_procedure + treatment) 

Looking at the p-values on treatment will tell you if the treatment was effective with a classical significance test.

Answer 2

Here is my attempt to make sense of this. (If this is not helpful, then perhaps a targeted revision of your question will receive more helpful attention.)

First, look at the first three months. A contingency table of Month by Proc (With or Without) is shown below, along with a chi-squared test to see if scheduling varied by month. No significant difference: P-value > 0.05.

DTA1
     [,1] [,2]
[1,]   55  388  
[2,]   58  385
[3,]   58  376

        Pearson's Chi-squared test

data:  DTA1
X-squared = 0.18624, df = 2, p-value = 0.9111

Similarly, for the last three months:

DTA2
     [,1] [,2]
[1,]   65  315
[2,]   67  302
[3,]   66  295

chisq.test(DTA2)

        Pearson's Chi-squared test

data:  DTA2
X-squared = 0.21354, df = 2, p-value = 0.8987

The P-values are so unusually high that I wonder whether scheduling may be systematic rather than random, with about the same number taken each month during the first three months (high 50's), and about the same number per month during last time period (high 60's).

Your main question seems to be whether scheduling behavior is different before and after intervention. So I compared combined results for the first three months with combined results from the last three months. There is a highly significant difference: a greater proportion of the patients of interest received the procedure during the last three months (almost 18%) than during the first three months (about 13%).

DTAc
  [,1] [,2]
a  171 1149
b  198  912

chisq.test(DTAc)

        Pearson's Chi-squared test with Yates' continuity correction

data:  DTAc
X-squared = 10.788, df = 1, p-value = 0.001022