Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 6 down vote favorite
1

The setup for this question is as follows. Lets say we have count and denominator data for six different study sites, by month for 12 months. They're all using a uniform diagnostic standard at the time, so theoretically their counts are all roughly the same. The goal of this is to get an estimate of the typical monthly case count per person-time each study site gets, though we allow it to vary by month in the event that it changes dramatically.

So essentially, a pretty standard Poisson regression estimation of an incidence-density.

Except two of the sites change their diagnostic procedure to a more accurate one somewhere in the middle of the study period. Lets say Site 1 changes 2 months in, and Site 2 changes 8 months in, for the sake of the example.

The new procedure is believed to be of improved accuracy, but there's no direct data from the sites to produce our own accuracy measures, we'd have to go off published sensitivity/specificity figures from the two tests in the literature.

I'm trying to figure out a way to get around just out and out discarding the data from after the protocol change. Any ideas?

An update: One of the sites in question is about 3% of the total study person-time but about 10% of the cases, both because of the better test, and by the looks of it, an outbreak at the site at the time. So while "toss it" is still an option, it's not my favorite option.

share|improve this question
Please don’t take this as an offense, but this reminds me the famous quotation: “To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of.” R. A. Fisher – Elvis Jan 7 '12 at 9:55
3  
@Elvis No offense taken. Though R.A. Fisher should count himself lucky for never having to work with surveillance data collected for other purposes. It's either messy data, or a clinician's gut instinct - I'll take what data I can get. – EpiGrad Jan 7 '12 at 20:10
I know... Don't tell me, some if my data are... it's even worst than that, sometimes no one can tell why yhey tried to replicate some results that weren't even suggestive the first time. – Elvis Jan 7 '12 at 21:29
Given that the new diagnostic procedure is more accurate, is it likely that the other sites will shift to it? So would it be better to estimate off the new method. I would worry that the other sites are producing an artificially deflated estimate. Do you have enough data to track both types of site separately? – Michelle Jan 25 '12 at 9:05
@Michelle They likely have already, but not in our time frame. And unfortunately no, it's just the one site so far in our data. I've filed tracking them separately off as a side project for the future. – EpiGrad Jan 25 '12 at 17:47

1 Answer

up vote 2 down vote accepted

based on the question and comments, can you make an adjustment so that you deflate the number of cases under the new protocol to what they "would" have had under the old protocol? If you know the difference in the specificity and the sensitivity for both diagnostic tests, and you know the number of patients that the two sites are testing, can you adjust their new figures on the basis of the differences between those two tests. You're not worried about "true" positives, just about the total number diagnosed. But you'll need the numbers of patients tested in order to make the correction.

One possible issue: did the two sites change over completely to the new diagnostic method, or did they run a combination of both methods (old method on some patients, new method on others) at the same time? If so, I can't think of a correction for that.

share|improve this answer
Unfortunately the answer seems to be "toss out the data" - we actually don't know the specificity and sensitivity, nor did any sites overlap. Just some rough clinical guidelines and a pretty informal survey. – EpiGrad Jan 25 '12 at 18:06
Just remember that you tried to resuscitate the patient! :) Can you add the information about the new method into your report, as I think the new information should be published - it's novel and seems to suggest an increase in case load, so it is of practical significance to clinicians. – Michelle Jan 25 '12 at 18:10
It's been filed as a side project when I finish my dissertation, but yeah, hopefully I'll be able to do something with it. – EpiGrad Jan 25 '12 at 18:12

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.