1
$\begingroup$

I am editing this question and including generated data that is similar to the data that I have from a clinic. We have Medical residents that want to look at a particular event and see what percentage of patients that are < 28 months from event and have a reversible device placed. Each month from Dec 2017 to Dec 2018 the Electronic health record was scanned looking back 27 months for particular event and device placement. Two interventions took place involving ease of access (April 2018) for particular device and educating staff (July 2018). They are looking to see if interventions had any significant bearing on increased usage. I am looking at how to set this up to test for significance. It was originally tested using t-tests of differences in average percentage of patients comparing 3 different months (Dec17-Feb18) to another 3 months (Oct18-Dec18). These 3 month blocks represent before and after interventions. The person that originally tested has moved and I was given the data. I think there is a flaw in the way the percentages were calculated. Each month had a lookback and usage was tallied, so from month to month the same patient was counted again if they met the condition of < 28 months from event. Plus the post intervention collection counted the same patients that were in the pre intervention months if they still met the < 28 month cutoff. I think either an interrupted time series or some pre post proportion test that did not count patients twice would be the way to go. Below is the data that simulates what is in the clinic. I had to do this for HIPAA reasons. Any help on this would be appreciated. The data is from r language.

set.seed(1)
id <- sample(100000:199999, 500, replace=FALSE)


Event_Date <- as.Date("2015-10-01") + 
  runif( 500, 
         max=as.integer( 
           as.Date( "2018-12-31") - 
             as.Date( "2015-10-01")))

sdf <- data.frame(id,Event_Date)
sdf <- sdf[order(as.Date(sdf$Event_Date)),]
rownames(sdf) <- NULL

fdr <- rbinom(n=396, size=1, prob=0.13)
sdr <- rbinom(n=40, size=1, prob=0.155)
tdr <- rbinom(n=64, size=1, prob=0.175)
sdf$Device <- c(fdr,sdr,tdr)



monnb <- function(d) { lt <- as.POSIXlt(as.Date(d, origin="1900-01-01")); lt$year*12 + lt$mon } 
# compute a month difference as a difference between two monnb's
mondf <- function(d1, d2) { monnb(d2) - monnb(d1) }

sdf$Dec17Scan <- ifelse((mondf(sdf$Event_Date,"2017-12-31")<28 & mondf(sdf$Event_Date,"2017-12-31") >= 0),1,0)
sdf$Jan18Scan <- ifelse((mondf(sdf$Event_Date,"2018-01-31")<28 & mondf(sdf$Event_Date,"2018-01-31") >= 0),1,0)
sdf$Feb18Scan <- ifelse((mondf(sdf$Event_Date,"2018-02-28")<28 & mondf(sdf$Event_Date,"2018-02-28") >= 0),1,0)
sdf$Mar18Scan <- ifelse((mondf(sdf$Event_Date,"2018-03-31")<28 & mondf(sdf$Event_Date,"2018-03-31") >= 0),1,0)
sdf$Apr18Scan <- ifelse((mondf(sdf$Event_Date,"2018-04-30")<28 & mondf(sdf$Event_Date,"2018-04-30") >= 0),1,0)
sdf$May18Scan <- ifelse((mondf(sdf$Event_Date,"2018-05-31")<28 & mondf(sdf$Event_Date,"2018-05-31") >= 0),1,0)
sdf$Jun18Scan <- ifelse((mondf(sdf$Event_Date,"2018-06-30")<28 & mondf(sdf$Event_Date,"2018-06-30") >= 0),1,0)
sdf$Jul18Scan <- ifelse((mondf(sdf$Event_Date,"2018-07-31")<28 & mondf(sdf$Event_Date,"2018-07-31") >= 0),1,0)
sdf$Aug18Scan <- ifelse((mondf(sdf$Event_Date,"2018-08-31")<28 & mondf(sdf$Event_Date,"2018-08-31") >= 0),1,0)
sdf$Sep18Scan <- ifelse((mondf(sdf$Event_Date,"2018-09-30")<28 & mondf(sdf$Event_Date,"2018-09-30") >= 0),1,0)
sdf$Oct18Scan <- ifelse((mondf(sdf$Event_Date,"2018-10-31")<28 & mondf(sdf$Event_Date,"2018-10-31") >= 0),1,0)
sdf$Nov18Scan <- ifelse((mondf(sdf$Event_Date,"2018-11-30")<28 & mondf(sdf$Event_Date,"2018-11-30") >= 0),1,0)
sdf$Dec18Scan <- ifelse((mondf(sdf$Event_Date,"2018-12-31")<28 & mondf(sdf$Event_Date,"2018-12-31") >= 0),1,0)

Thanks for any help guiding what test to use.

|cite|improve this question
$\endgroup$
  • $\begingroup$ Are individuals in the three row observational period double-counted? e.g. are the 443 patients on Dec-17 the same as the 443 patients labeled on Jan-18? Same goes with the other columns $\endgroup$ – Parseltongue Jul 24 at 0:49
  • $\begingroup$ I am trying to get raw data, but from what was told it is cumulative across months and therefore duplication. I think a report was run from the EHR and if they fit the < 27 month and had procedure in all months then they were counted. So potentially in Dec 2017 you could have 20 of 443 patients at month 25 that are in Jan 2018 as well, then fall off in month Feb 2018 and pick up new patients along the way. If there is a better way of counting, I am all ears. $\endgroup$ – Brad Jul 24 at 2:36
  • $\begingroup$ I think the better way of counting is by having the data at the patient level where each row represents a unique patient. You then track whether that specific patient had the procedure or not. I might just be misunderstanding $\endgroup$ – Parseltongue Jul 24 at 2:46
  • $\begingroup$ @Parseltongue see post edit, Thanks for your help. $\endgroup$ – Brad Jul 24 at 4:00
  • $\begingroup$ Hey Brad, I'm sorry, I just am having a hard time visualizing your dataset. I'd wait until somebody else responds. For the time being, a t-test like you're describing will suffice. It's definitely problematic to include participants 0 months from procedure and 27 months prior, but I don't know how to deal with that $\endgroup$ – Parseltongue Jul 24 at 19:20
0
$\begingroup$

I would create a dummy variable that is equal to 1 for all values after the "treatment" and is equal to 0 for all values before the treatment. It's sloppy, but you can collapse across the months by taking the average value of the "Pts < 27" column and transforming that as a variable called percent_with_procedure.

Then run a regression of the form:

lm(percent_with_procedure + treatment)

Looking at the p-values on treatment will tell you if the treatment was effective with a classical significance test.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Trying to measure if more patients are on treatment based on access barriers. The intervention is better access and staff education. $\endgroup$ – Brad Jul 24 at 19:04
0
$\begingroup$

Here is my attempt to make sense of this. (If this is not helpful, then perhaps a targeted revision of your question will receive more helpful attention.)

First, look at the first three months. A contingency table of Month by Proc (With or Without) is shown below, along with a chi-squared test to see if scheduling varied by month. No significant difference: P-value > 0.05.

DTA1
     [,1] [,2]
[1,]   55  388  
[2,]   58  385
[3,]   58  376

        Pearson's Chi-squared test

data:  DTA1
X-squared = 0.18624, df = 2, p-value = 0.9111

Similarly, for the last three months:

DTA2
     [,1] [,2]
[1,]   65  315
[2,]   67  302
[3,]   66  295

chisq.test(DTA2)

        Pearson's Chi-squared test

data:  DTA2
X-squared = 0.21354, df = 2, p-value = 0.8987

The P-values are so unusually high that I wonder whether scheduling may be systematic rather than random, with about the same number taken each month during the first three months (high 50's), and about the same number per month during last time period (high 60's).

Your main question seems to be whether scheduling behavior is different before and after intervention. So I compared combined results for the first three months with combined results from the last three months. There is a highly significant difference: a greater proportion of the patients of interest received the procedure during the last three months (almost 18%) than during the first three months (about 13%). 

DTAc
  [,1] [,2]
a  171 1149
b  198  912

chisq.test(DTAc)

        Pearson's Chi-squared test with Yates' continuity correction

data:  DTAc
X-squared = 10.788, df = 1, p-value = 0.001022
|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.